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SECTION  1.  GENERAL  INFORMATION 


1.1  BACKGROUND 

Technologies  under  development  for  the  detection  and  discrimination  of  military 
munitions  (MM)  (i.e.  unexploded  ordnance  {UXO}  and  discarded  military  munitions  {DMM}) 
require  testing  so  that  performance  can  be  characterized.  To  that  end,  Standardized  Test  Sites 
have  been  developed  at  Aberdeen  Proving  Ground  (APG),  Maryland,  and  U.S.  Army  Yuma 
Proving  Ground  (YPG),  Arizona.  These  test  sites  provide  a  diversity  of  geology,  climate, 
terrain,  and  weather  as  well  as  diversity  in  munitions  and  clutter.  Testing  at  these  sites  is 
independently  administered  and  analyzed  by  the  government  for  the  purposes  of  characterizing 
technologies,  tracking  perfonnance  with  system  development,  comparing  performance  of 
different  systems,  and  comparing  performance  in  different  environments. 

The  Standardized  UXO  Technology  Demonstration  Site  Program  is  a  multiagency 
program  spearheaded  by  the  U.S.  Army  Environmental  Command  (USAEC).  The  U.S.  Army 
Aberdeen  Test  Center  (ATC)  and  the  U.S.  Army  Corps  of  Engineers  Engineering  Research  and 
Development  Center  (ERDC)  provide  programmatic  support.  The  program  is  being  funded  and 
supported  by  the  Environmental  Security  Technology  Certification  Program  (ESTCP),  the 
Strategic  Environmental  Research  and  Development  Program  (SERDP),  and  the  U.S.  Army 
Environmental  Quality  Technology  (EQT)  Program. 

1.2  SCORING  OBJECTIVES 

The  objective  in  the  Standardized  UXO  Technology  Demonstration  Site  Program  is  to 
evaluate  the  detection  and  discrimination  capabilities  of  a  given  technology  under  various  field 
and  soil  conditions.  Inert  munitions  and  clutter  items  are  positioned  in  various  orientations  and 
depths  in  the  ground. 

The  evaluation  objectives  are  as  follows: 

a.  To  determine  detection  and  discrimination  effectiveness  under  realistic  scenarios  with 
various  targets,  geology,  clutter,  density,  topography,  and  vegetation. 

b.  To  determine  cost,  time,  and  workforce  requirements  to  operate  the  technology. 

c.  To  determine  the  demonstrator’s  ability  to  analyze  survey  data  in  a  timely  manner  and 
provide  prioritized  Target  Lists  with  associated  confidence  levels. 

d.  To  provide  independent  site  management  to  enable  the  collection  of  high  quality, 
ground-truth  (GT),  geo-referenced  data  for  post-demonstration  analysis. 
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1.2.1  Scoring  Methodology 

a.  The  scoring  of  the  demonstrator’s  performance  is  conducted  in  two  stages:  response 
stage  and  discrimination  stage.  For  both  stages,  the  probability  of  detection  (Pd)  and  the  false 
alarms  are  reported  as  receiver-operating  characteristic  (ROC)  curves.  False  alarms  are  divided 
into  those  anomalies  that  correspond  to  emplaced  clutter  items,  measuring  the  probability  of 
clutter  detection  (Pcd)  or  the  probability  of  false  positive  (Pfp).  Those  that  do  not  correspond  to 
any  known  item  are  tenned  background  alarms.  The  background  alanns  are  addressed  as  either 
probability  of  background  alarm  (Pba)  or  background  alarm  rate  (BAR). 

b.  The  response  stage  scoring  evaluates  the  ability  of  the  system  to  detect  emplaced 
targets  without  regard  to  ability  to  discriminate  munitions  from  other  anomaly  sources.  For  the 
blind  grid  response  stage,  the  demonstrator  provides  a  target  response  from  each  and  every  grid 
square  along  with  a  threshold  below  which  target  responses  are  deemed  insufficient  to  warrant 
further  investigation.  This  list  is  generated  with  minimal  processing  and,  since  a  value  is 
provided  for  every  grid  square,  includes  amplitudes  both  above  and  below  the  system  noise  level. 
For  the  open  field,  the  demonstrator  provides  a  list  of  all  anomalies  deemed  to  exceed  a 
demonstrator  selected  target  detection  threshold.  An  item  (either  munition  or  clutter)  is  counted 
as  detected  if  a  demonstrator  indicates  an  anomaly  within  a  specified  distance  (Halo  Radius 
(Rhaio))  of  a  ground  truth  item. 

c.  The  discrimination  stage  evaluates  the  demonstrator’s  ability  to  correctly  identify 
munitions  as  such  and  to  reject  clutter.  For  the  blind  grid  discrimination  stage,  the  demonstrator 
provides  the  output  of  the  discrimination  stage  processing  for  each  grid  square.  For  the  open 
field,  the  demonstrator  provides  the  output  of  the  discrimination  stage  processing  for  anomaly 
reported  in  the  response  stage.  The  values  in  these  lists  are  prioritized  based  on  the 
demonstrator’s  determination  that  a  location  is  likely  to  contain  munitions.  Thus,  higher  output 
values  are  indicative  of  higher  confidence  that  a  munitions  item  is  present  at  the  specified 
location.  For  digital  signal  processing,  priority  ranking  is  based  on  algorithm  output.  For  other 
discrimination  approaches,  priority  ranking  may  be  based  on  rule  sets  or  human  judgment.  The 
demonstrator  also  specifies  the  threshold  in  the  prioritized  ranking  that  provides  optimum 
performance,  (i.e.,  that  is  expected  to  retain  all  detected  munitions  and  reject  the  maximum 
amount  of  clutter). 

d.  The  demonstrator  is  also  scored  on  efficiency  and  rejection  ratios,  which  measure  the 
effectiveness  of  the  discrimination  stage  processing.  The  goal  of  discrimination  is  to  retain  the 
greatest  number  of  munitions  detections  from  the  anomaly  list,  while  rejecting  the  maximum 
number  of  anomalies  arising  from  nonmunitions  items.  Efficiency  measures  the  fraction  of 
detected  munitions  retained  after  discrimination,  while  the  rejection  ratio  measures  the  fraction 
of  false  alanns  rejected.  Both  measures  are  defined  relative  to  the  maximum  number  of 
munitions  detectable  by  the  sensor  and  its  accompanying  clutter  detection/false  positive  rate  or 
BAR. 
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e.  Based  on  configuration  of  the  GT  at  the  standardized  sites  and  the  defined  scoring 
methodology,  in  some  cases,  there  exists  the  possibility  of  having  anomalies  within  overlapping 
halos  and/or  multiple  anomalies  within  halos.  In  these  cases,  the  following  scoring  logic  is 
implemented: 

(1)  In  situations  where  multiple  anomalies  exist  within  a  single  Rhaio,  the  anomaly  with 
the  strongest  response  or  highest  ranking  will  be  assigned  to  that  particular  GT  item.  If  the 
responses  or  rankings  are  equal,  then  the  anomaly  closest  to  the  GT  item  will  be  assigned  to  the 
GT  item.  Remaining  anomalies  are  retained  and  scored  until  all  matching  is  complete. 

(2)  Anomalies  located  within  any  Rhai0  that  do  not  get  associated  with  a  particular  GT 
item  are  excess  alanns  and  will  be  disregarded. 

f.  In  some  cases,  groups  of  closely  spaced  munitions  have  overlapping  halos.  The 
following  scoring  logic  is  implemented  (fig.  A-l  through  A-9): 

(1)  Overall  site  scores  (i.e.,  Pd)  will  consider  only  isolated  munitions  and  clutter  items. 

(2)  GT  items  that  have  overlapping  halos  (both  munitions  and  clutter)  will  form  a  group 
and  groups  may  form  chains. 

(3)  Groups  will  have  a  complex  halos  composed  of  the  composite  halos  of  all  its  GT 

items. 

(4)  Groups  will  have  three  scoring  factors:  groups  found,  groups  identified,  and  group 
coverage.  Scores  will  be  based  on  1 : 1  matches  of  anomalies  and  GT. 

(a)  Groups  Found  (Found):  the  number  of  groups  that  have  one  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  detecting  a 
group  if  any  item  within  the  group  is  matched  to  an  anomaly  in  their  lists. 

(b)  Groups  Identified  (ID):  the  number  of  groups  that  have  two  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  identifying 
that  a  group  is  present  if  multiple  items  within  the  composite  halo  are  matched  to  anomalies  in 
their  lists. 

(c)  Group  Coverage  (Coverage):  the  number  of  GT  items  matched  within  groups  divided 
by  the  total  number  of  GT  items  within  groups.  This  metric  measures  the  demonstrator  accuracy 
in  determining  the  number  of  anomalies  within  a  group.  If  five  items  are  present  and  only  two 
anomalies  are  matched,  the  demonstrator  will  score  0.4.  If  all  five  are  matched,  the  demonstrator 
will  score  1.0. 

(5)  Location  error  will  not  be  reported  for  groups. 
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(6)  Demonstrators  will  not  be  asked  to  call  out  groups  in  their  scoring  submissions.  If 
multiple  anomalies  are  indicated  in  a  small  area,  the  demonstrator  will  report  all  individual 
anomalies. 

(7)  Excess  alarms  within  a  halo  will  be  disregarded. 

g.  All  scoring  factors  are  generated  utilizing  the  Standardized  UXO  Probability  and  Plot 
Program,  version  4. 

1.2.2  Scoring  Factors 

Factors  to  be  measured  and  evaluated  as  part  of  this  demonstration  include: 

a.  Response  stage  ROC  curves: 

(1)  Probability  of  detection  (Pdres). 

(2)  Probability  of  clutter  detection  (Pcd). 

(3)  Background  alann  rate  (BARres)  or  probability  of  background  alarm  (Pbares). 

b.  Discrimination  stage  ROC  curves: 

(1)  Probability  of  detection  (Pddlsc). 

(2)  Probability  of  false  positive  (Pfp). 

(3)  Background  alann  rate  (BARdlsc)  or  probability  of  background  alarm  (Pbadlsc). 

c.  Metrics: 

(1)  Efficiency  (E). 

(2)  False  positive  rejection  rate  (Rfp). 

(3)  Background  alarm  rejection  rate  (Rba). 

d.  Other: 

(1)  Probability  of  detection  by  size,  depth,  and  density. 

(2)  Classification  by  type  (i.e.,  20-,  40-,  105mm,  etc.). 

(3)  Location  accuracy  for  single  munitions. 
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(4)  Equipment  setup,  calibration  time,  and  corresponding  worker-hour  requirements. 

(5)  Survey  time  and  corresponding  worker-hour  requirements. 

(6)  Reacquisition/resurvey  time  and  worker-hour  requirements  (if  any). 

(7)  Downtime  due  to  system  malfunctions  and  maintenance  requirements. 
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SECTION  2.  DEMONSTRATION 


2.1  DEMONSTRATOR  INFORMATION 

2.1.1  Demonstrator  Point  of  Contact  (POC)  and  Address 

POC:  Mr.  David  George 

970  263-9714 

Address:  G&G  Sciences,  Inc. 

23  Road 

Grand  Junction,  CO  81505 

2.1.2  System  Description  (provided  by  demonstrator) 

Under  development  since  2003,  the  Advanced  Ordnance  Locator  (AOL)  system  is  a 
dual-mode  (electromagnetic/magnetometer  (EM/MAG))  system  for  UXO  detection  and 
characterization.  The  prototype  AOL  system  was  developed  under  contract  to  Naval  Explosive 
Ordnance  Disposal  Technology  Division  (NAVEODTECHDIV)  (Indian  Head)  by  Blackhawk 
GeoServices  (now  Zapata  Blackhawk)  with  Geometries  and  G&G  Sciences,  Inc.  acting  as 
subcontractor.  In  2006,  G&G  Sciences,  Inc.  received  a  follow-on  contract  to  continue  the 
development  of  the  AOL  system.  As  a  platform  for  electromagnetic  interference  (EMI)  research, 
the  AOL2  system  is  unique  and  innovative  in  several  respects: 

a.  Multiple  Transmitter  Loops:  The  AOL2  antenna  platfonn  includes  three  mutually 
orthogonal  transmitter  loops. 

b.  3-Axis  Sensor  Array:  The  AOL2  antenna  platform  includes  a  spatial  array  of  nine 
3 -axis  receiver  antennas  (27  independent  measurements  of  the  secondary  magnetic  field). 

c.  Electronically  Switched  Time-Gated  Electromagnetic  (TEM)  Transmitter  Loop 
Driver:  The  AOL2  system  is  unique  in  its  ability  to  drive  its  transmitter  loop  array.  Under 
control  of  the  data  acquisition  (DAQ)  computer,  the  output  of  the  transmitter  can  be  directed  to 
any  single  loop  or  automatically  multiplexed  between  loops.  There  is  also  control  of  the 
fundamental  waveform  period,  duty-cycle,  and  pulse  polarity.  Typically,  however,  the  loops  are 
driven  with  a  classical  bipolar  pulse  type  TEM  waveform  (i.e.,  alternating  pulse  polarity  with  a 
50  percent  duty-cycle.  Depending  on  the  survey  mode  (e.g.,  static/dynamic),  the  fundamental 
frequency  of  transmission  can  be  varied  from  a  low  of  1.11  </<  8 10  Hz. 
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Figure  1 .  Demonstrator’s  system,  AOL/dual  mode. 


2.1.3  Data  Processing  Description  (provided  by  demonstrator) 

Acquisition  Modes. 

a.  The  AOL2  system  is,  by  design,  a  very  flexible  system  for  acquisition  of  time  domain 
EM  (TEM)  data.  It  is  beyond  the  scope  of  this  document  to  fully  describe  that  flexibility.  Simply 
stated,  data  are  acquired  in  time  blocks  that  consist  of  a  fixed  number  of  transmitter  cycle 
repeats.  Both  the  period  (T)  and  the  repeat  factor  (N)  are  operator  selectable  and  are  varied  in 
multiplicative  factors  of  three.  It  has  two  DAQ  modes: 

(1)  Static-mode  acquisition:  In  this  mode,  data  sampled  transients  from  each  of  the 
27  receiver  loops  plus  a  channel  to  sense  the  transmitter  loop  current  are  rectified  and  stacked  for 
a  specified  number  of  acquisition  blocks.  The  resulting  transients  are  (optionally)  decimated  into 
a  set  of  logarithmically  spaced  time  gates,  after  which  they  are  stored  to  a  single  binary  data  file. 
As  its  name  implies,  static-mode  acquisition  is  used  to  obtain  precise  data  while  the  antenna 
platform  is  parked  at  a  single  spatial  data  point. 

(2)  Continuous-mode  acquisition:  As  its  name  implies,  continuous-mode  data  acquisition 
results  in  the  DAQ  cycle  being  repeated  until  the  operator  intervenes  to  halt  it.  Each  of  the  data 
points  are  appended  to  single  binary  data  file  and  thus  the  resulting  data  file  may  consists  of  10s 
or  even  100s  of  data  points.  This  mode  is  used  for  dynamic  surveying.  Typically,  a  data  file 
consists  of  all  the  points  acquired  along  a  single  profile. 
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b.  Regardless  of  the  acquisition  mode,  the  TEM  data  thus  acquired  includes  the  most 
current  Global  Positioning  System  (GPS)  position  and  the  platform  attitude  angles  (magnetic 
heading,  pitch,  and  roll).  Depending  on  the  block  period  (T)  and  the  repeat  factor  (N),  sampling 
rates  as  high  as  30  samples/sec  can  be  achieved.  As  we  have  stated  above,  the  data  are  stored  as 
binary  fonnatted  files.  However,  our  processing  software  includes  the  capability  to  export  the 
data  to  a  Geosoft  Oasis  Montaj  data  base  for  further  quality  control  (QC)  and  map  compilation. 
The  processing  also  includes  the  capability  to  export  the  data  to  text  files. 

Target  Selection.  G&G  Sciences,  Inc.  plan  is  to  complete  dynamic  surveys  over  both  the 
calibration  and  blind  grids.  The  surveys  will  consist  of  parallel  profiles  acquired  with  1 -meter 
offsets.  Using  these  data,  a  detection  parameter  map  of  the  surveyed  area  will  be  compiled.  The 
detection  map  is  based  on  the  magnitude  of  the  secondary  fields  measured  at  each  of  the  nine 
tri-axial  receiver  sensors.  The  following  processing  steps,  accomplished  using  Geosoft  Oasis 
Montaj™,  are  required: 

a.  Metal  Mapper  data  are  recorded  as  binary  files.  These  data  are  imported  directly  into 
an  OM  data  base  where  simple  editing  (e.g.,  editing  line  numbers,  deselecting  duplicate  lines, 
trimming  and  deleting  bad  data  or  stops,  etc).  All  other  steps  are  accomplished  from  within  OM 
using  its  standard  editing  and  processing  capabilities  supplemented  where  necessary  with  custom 
Geosoft  Executables  (GXs),  Geosoft  Scripts  (GSs),  and  Geosoft  mathematical  expression  (EXP) 
files. 

b.  Convert  Lat/Lon  to  UTM  coordinates. 

c.  Compute  detector  gate  values  for  each  of  the  27  receiver  channels. 

d.  Normalize  detector  gate  values  by  transmitter  current. 

e.  Select  background  and  remove  background  (leveling). 

f.  Generate  vector  magnitude  channels  for  each  of  nine  tri-axial  receiver  cubes 

g.  Make  heading  channel  for  each  profile. 

h.  Split  each  profile  into  nine  separate  profiles,  corrected  for  heading  and  offset  distance 
from  the  platfonn  measure  point  (generates  nine  parallel  profiles  with  1 1-cm  offsets). 

i.  Grid  cube  amplitude  data. 

j.  Apply  grid  smoothing  filters  if  necessary. 

k.  Select  targets  using  an  amplitude  threshold.  The  (tunable)  parameters  are: 

(1)  Signal  amplitude. 

(2)  Detector  gate  (step  3). 
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1.  Edit  target  list  based  on  inspection  of  profiles. 

Target  Reacquisition  and  Parameter  Estimation.  Each  of  the  targets  generated  from  the 
detection  map  created  from  the  dynamic  data  are  reacquired  with  the  Metal  Mapper  using  a 
combination  of  GPS  to  return  to  the  approximate  target  location  and  then  a  real-time  graphics 
display  that  allows  the  operator  to  center  the  antenna  platform  directly  over  the  target.  Once  the 
target  has  been  reacquired,  a  static  data  set  is  acquired  at  that  position.  In  its  static  acquisition 
mode,  all  three  transmitter  loops  are  energized  in  turn.  Typically,  a  static  data  set  will  consist  of 
a  stack  of  50  to  100  data  blocks  and  the  acquisition  parameters  are  selected  so  that  8.33  ms  or 
25  ms  transients  are  acquired.  These  data  are  recorded  in  the  same  standard  binary  format  as  is 
the  dynamic  data.  However,  each  data  file  includes  only  a  single  (stacked)  data  point  rather  than 
a  sequence  of  data  points  that  are  stored  in  a  data  file  recorded  in  the  continuous  acquisition 
mode.  Each  of  the  static  data  files  are  used  as  input  to  the  program  (TEMDipole).  TEMDipole 
is  a  physics-based  inversion  program  that  approximates  the  transient  response  of  compact 
metallic  objects  with  a  point  dipole  characterized  by  a  time-varying  anisotropic  polarizability 
tensor.  The  program  provides  optimum  estimates  of  the  following  parameters: 

a.  Target  Position  (x,  y,  and  z):  The  3-dimensional  position  of  the  target  with  respect  to 
the  position  of  the  antenna  platfonn.  The  Metal  Mapper  includes  an  apparatus  that  senses  the 
platform  attitude  angles  (heading,  pitch,  and  roll).  Thus  the  target  position  relative  to  the 
platform  coordinate  system  can  be  converted  to  geographic  coordinates. 

b.  Target  Attitude  (heading,  pitch,  roll):  The  Metal  Mapper  Inversion  software  estimates 
the  target  attitude  by  finding  the  principal  coordinate  system  for  the  target  polarizability. 

c.  Principal  Polarizability  Transients  (PI,  P2,  P3):  The  Metal  Mapper  Inversion  software 
estimates  the  three  principal  polarizability  transients  for  the  target.  Examples  of  the 
polarizability  curves  estimated  by  three  different  programs  using  a  static  data  set  collected  with 
the  AOL  at  YPG  in  2007. 

d.  The  nine  parameters  enumerated  previously  together  with  the  inversion  fit  statistics  are 
the  fundamental  data  derived  from  the  TEMDipole  inversion,  particularly,  the  principal 
polarizability  transients  such  as  those  containing  information  about  the  target.  For  example,  if 
both  targets  are  elongated  and  exhibit  a  single  axis  of  symmetry  as  indicated  by  the  fact  that 
there  is  a  single  major  polarizability  transient  and  two  nearly  identical  minor  polarizability 
curves,  a  measure  of  target  size  is  provided  by  the  integration  beneath  the  polarizability  curves. 
Note  that  the  units  of  the  polarizability  (rate)  transients  are  m  /s,  or,  equivalently  cm  /ps.  When 
integrated  over  time  to  find  the  area  beneath  the  curve,  we  end  up  with  units  of  volume 
(m3  or  cm3)  as  shown  in  the  formula  below: 


P  0  = 


R(t  =  0)  =  J 

o 


dP{t ) 
dt 


-dt 
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e.  G&G  Sciences,  Inc.  uses  the  root  mean  square  (RMS)  value  of  the  three  Pos  that  can 
calculate  from  the  three  principal  polarizability  transients  that  characterized  each  of  the  targets  as 
an  indication  of  size.  The  parameter  Po  defined  in  the  equation  is  an  example  of  a  so-called 
metaparameter  that  can  be  derived  from  the  more  fundamental  target  data  that  are  the  three 
principal  polarizability  curves.  For  simple  classification  by  shape,  one  can  define  other 
meta-parameters  involving  the  relationship  of  the  three  integrated  polarizability  parameters  (Pox, 
P Oyt  and  P(t)  derived  from  the  equation  to  identity  elongate  targets  with  an  axis  of  symmetry. 
Such  target  features  have  been  used  effectively  by  many  to  develop  classification  metrics  (1,  2). 
Among  the  more  useful  parameters  are  the  following: 

(1)  Transverse  polarizability:  Pqt  =  ( Poy  +  Poz)/2 

(2)  Polarizability  ratio:  Rpo  =  Pqx/Pot 

(3)  Eccentricity:  EPO  =  \P0y  -  Po2\/Pox 

f.  Generally  speaking,  UXO  have  a  polarizability  ratio  RPo  >  1  and  an  eccentricity 
EPo«  1  indicating  an  elongate  body  with  an  axis  of  symmetry.  The  thresholds  of 
discrimination  for  a  classifier  are  detennined  using  a  set  of  training  parameters  derived  from  a 
data  set  for  which  the  GT  is  known  (e.g.,  the  calibration  lanes). 

g.  Using  the  training  data,  a  classifier  is  developed  based  on  principles  of  pattern 
recognition  using  the  two  or  three  most  significant  parameters.  Typically,  the  classifier  is  based 
on  the  searching  of  the  nearest  neighbors  in  order  to  find  the  (binary)  decision  boundary 
providing  the  best  division  between  ordnance  (O)  and  clutter  (C).  To  facilitate  the  development 
of  a  classifier  for  a  particular  data  set,  we  use  the  Duke  Pattern  Recognition  Toolbox  (DPRT),  a 
library  of  MatLab  functions  for  pattern  recognition  developed  by  Leslie  Collins  and  her 
colleagues  at  Duke  University.  DPRT  supports  the  development  of  a  variety  of  classifiers 
including  kNN  (‘k’  nearest  neighbors)  and  FLD  (Fisher  Linear  Discriminant).  In  our  limited 
experience,  the  kNN  classifier  (with  k  =  3)  does  better  than  the  FLD  classifier  and  the  two.  The 
two  parameters  are  the  eccentricity  ( E)  and  the  polarizability  ratio  (R).  The  results  from  the  kNN 
classifier  are  effective  at  discriminating  between  loops  and  other  targets  with  good  symmetry. 
However,  there  is  no  basis  from  this  data  set  to  discriminate  the  shot  puts  from  other  targets. 
Indeed,  the  AOL2  polarizability  results  show  that  a  number  of  target  types  such  as  the  M75, 
MK118  Rockeye,  and  BLU-26  exhibit  three  nearly  identical  principal  polarizability  curves  thus 
indicating  near  isotropic  polarizability.  However,  the  shapes  of  the  principal  polarizabilities  for 
each  of  the  targets  are  distinctly  different. 

h.  Training.  The  performance  of  the  classifier  is  very  much  dependent  both  on  the  quality 
of  the  training  data  set  and,  as  well,  on  the  choice  of  the  relevant  parameters  used  in  training.  As 
of  yet,  there  is  no  feedback  on  the  performance  of  the  classifier  as  applied  to  a  similar  data  set 
acquired  over  the  blind  test  grid.  But  generally  speaking,  it  is  believed  that  the  training  data 
from  the  YPG  calibration  grid  is  flawed  in  the  sense  that  none  of  the  targets  in  the  calibration 
lanes  are  truly  clutter.  The  main  objective  of  the  work  that  is  planned  at  APG  is  to  conduct 
experiments  aimed  at  improving  the  ability  to  detect  deep  targets.  However,  a  target  list  must  be 
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submitted  to  ATC  for  targets  that  have  been  identified  within  the  blind  grid  for  scoring.  In  that 
regard,  either  a  dKNN  or  perhaps  a  neural  net  classifier  will  be  applied  to  the  appropriate  target 
parameters  extracted  from  static  measurements  over  the  cells  in  the  blind  grid  that  are  identified 
as  target  cells  from  the  dynamic  detection  survey. 

Parameter  Estimation. 

a.  Which  characteristics  will  be  extracted  from  each  detected  item  and  input  to  the 
discrimination  algorithm  (e.g.  depth,  size,  polarizability  coefficients,  fit  quality,  etc)? 

For  the  surveys  done  by  the  AOL,  no  discrimination  was  done,  nor  was  it  intended  to 
be  done.  In  order  to  satisfy  the  requirements  of  the  submittal  to  ATC,  every  target  was  given  a 
simple  ranking: 

0,  a  blank  cell. 

1 ,  a  clutter  target. 

2,  an  ordnance  target. 

This  ranking  was  based  on  whether,  in  the  opinion  of  the  analyst,  the  cell  was  blank, 
contained  clutter,  or  contained  an  ordnance  item.  This  opinion  was  based  on  the  size  of  the 
anomaly  and  occasionally  on  whether  the  target  was  in  the  vicinity  of  other  targets  (like  clutter) 
or  was  possibly  a  misplaced  ordnance  item  or  a  misplaced  clutter  item.  In  the  final  analysis,  this 
ranking  is  speculative  and  is  not  intended  to  be  used  for  discrimination. 

b.  Why  have  these  characteristics  been  chosen  and  not  others  (e.g.,  empirical  evidence  of 
their  ability  to  help  discriminate,  inclusion  in  a  theoretical  tradition,  etc)?  Vendor  did  not 
address  in  questionnaire  submittal. 

c.  How  are  these  characteristics  estimated  (e.g.,  least-mean-squares  fit  to  a  dipole  model, 
etc.),  to  include  the  equations  that  are  used  for  parameter  estimation?  Vendor  did  not  address  in 
questionnaire  submittal. 

d.  What  tunable  parameters  (if  any)  are  used  in  the  characterization  process  (e.g., 
thresholds  on  background  noise,  etc.)?  Vendor  did  not  address  in  questionnaire  submittal. 

Classification. 

a.  What  algorithm  is  used  for  discrimination  (e.g.,  multilayer  perception,  support  vector 
machine,  etc.)?  Vendor  did  not  address  in  questionnaire  submittal. 

b.  Why  is  this  algorithm  used  and  not  others?  Vendor  did  not  address  in  questionnaire 
submittal. 

c.  Which  parameters  are  considered  as  possible  inputs  to  the  algorithm?  Vendor  did  not 
address  in  questionnaire  submittal. 
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d.  What  are  the  outputs  of  the  algorithm  (probabilities,  confidence  levels)?  Vendor  did 
not  address  in  questionnaire  submittal. 

e.  How  is  the  threshold  set  to  decide  where  the  munitions/nonmunitions  line  lies  in  the 
discrimination  process?  Vendor  did  not  address  in  questionnaire  submittal. 

Training. 

a.  Which  tunable  parameters  have  final  values  that  are  optimized  over  a  training  set  of 
data  and  which  have  values  that  are  set  according  to  geophysical  knowledge  (i.e.,  intuition, 
experience,  common  sense)?  Vendor  did  not  address  in  questionnaire  submittal. 

(1)  For  those  tunable  parameters  with  final  values  set  according  to  geophysical 
knowledge: 

(a)  What  is  the  reasoning  behind  choosing  these  particular  values?  Vendor  did  not 
address  in  questionnaire  submittal. 

(b)  Why  were  the  final  values  not  optimized  over  a  training  set  of  data?  Vendor  did  not 
address  in  questionnaire  submittal. 

(2)  For  those  tunable  parameters  with  final  values  optimized  over  the  training  set  of  data: 

(a)  What  training  data  is  used  (e.g.,  all  data,  a  randomly  chosen  portion  of  data, 
etc.)?  Vendor  did  not  address  in  questionnaire  submittal. 

(b)  What  error  metric  is  minimized  during  training  (e.g.,  mean  squared  error, 
etc.)?  Vendor  did  not  address  in  questionnaire  submittal. 

(c)  What  learning  rule  is  used  during  training  (e.g.,  gradient  descent,  etc.)?  Vendor  did 
not  address  in  questionnaire  submittal. 

(d)  What  criterion  is  used  to  stop  training  (e.g.,  number  of  iterations  exceeds  threshold, 
good  generalization  over  validation  set  of  data,  etc.)?  Vendor  did  not  address  in  questionnaire 
submittal. 

(e)  Are  all  tunable  parameter  optimized  at  once  or  in  sequence  (in  sequence  =  parameter  1 
is  held  constant  at  some  common  sense  values  while  parameter  2  is  optimized,  and  then 
parameter  2  is  held  constant  at  its  optimized  value  while  parameter  1  is  optimized)?  Vendor  did 
not  address  in  questionnaire  submittal. 

b.  What  are  the  final  values  of  all  tunable  parameters  for  the  characterization 
process?  Vendor  did  not  address  in  questionnaire  submittal. 
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2.1.4  Data  Submission  Format 


Data  were  submitted  for  scoring  in  accordance  with  data  submission  protocols  outlined  on 
the  USAEC  Web  site  www.uxotestsites.org.  These  submitted  data  are  not  included  in  this  report 
in  order  to  protect  GT  information. 

2.1.5  Demonstrator  Quality  Assurance  (QA)  and  Quality  Control  IOC)  (provided  by 

demonstrator) 

Quality  Control  (QC).  The  A0L2  DAQ  system  integrates  data  acquired  from  three 
(optionally  4)  sensors  into  a  sample  data  point.  These  systems  are  position;  attitude;  EM,  and 
(optionally)  MAG.  The  data  from  each  of  the  systems  are  integrated  into  a  single  data  structure 
(i.e.,  an  EM3DDataPoint).  Perfonned  system  checks  by  returning  to  a  calibration  point  to 
acquire  data  will  occur.  Typically,  the  system  check  consists  of  a  short  profile  (approximately 
1 0  m)  that  is  surveyed  repeatedly  two  or  more  times  a  day.  The  profile  will  be  set  up  in  an  area 
of  typical  background  response  (i.e.,  no  targets).  The  calibration  survey  will  consist  of  a 
dynamic  survey  run  over  a  calibration  target  (typically  a  shot  put)  centered  along  the  profile.  At 
the  start  of  the  calibration  survey,  a  static  point  using  both  dynamic  and  static  acquisition 
parameters  at  the  beginning  of  the  calibration  line  is  acquired,  the  target  is  surveyed  dynamically 
in  one  direction,  and  then  the  survey  is  repeated  in  the  opposite  direction.  Finally,  the  antenna 
array  is  halted  directly  over  the  target  and  acquires  a  static  data  point.  The  static  points 
(static/dynamic  parameters)  provide  base-level  background  measurements.  These  measurements 
are  useful  in  determining  whether  the  background  changes  significantly  over  the  area  of  the 
survey.  The  calibration  survey  lines,  repeated  in  opposite  directions,  provide  a  check  of  survey 
timing  latency  between  the  acquisition  of  the  GPS  position  and  the  acquisition  of  the  EM  data. 
Position  latencies  typical  of  systems  where  survey  positions  and  data  are  merged  from 
independent  data  files  based  on  a  time  stamp  have  not  been  experienced  because  of  the  way  the 
GPS  position  is  integrated  directly  with  the  data.  However,  this  experiment  provides 
proof-positive  that  there  is  no  significant  timing  latency  in  the  acquisition  system.  The 
amplitude  of  the  dynamic  survey  peaks  as  they  cross  over  the  calibration  target  and  also  provides 
a  crude  measure  of  the  EM  drift.  A  better  measure  of  the  drift  is  provided  by  the  static 
measurements  of  the  background  and  the  target  response.  As  part  of  the  static  background 
measurement,  a  precise  method  for  putting  the  cart  into  a  known  and  repeatable  attitude  will  be 
established  so  that  the  reliability  of  the  orientation  system  may  be  checked.  It  is  notable  that  the 
DAQ  system  constantly  monitors  the  quality  of  the  GPS  positions  and  provides  a  visual  warning 
to  the  operator  when  the  GPS  quality  for  any  reason  degrades  below  that  of  real-time  kinematic 
(RTK).  Furthermore,  the  acquisition  software  includes  the  ability  to  graphically  display  data 
from  any  point  in  any  data  file.  This  plotting  capability  allows  the  data  to  be  checked  at  anytime 
while  in  the  field. 

Overview  of  Quality  Assurance  (QA). 

a.  The  single  objective  for  the  work  planned  at  APG  is  to  use  the  Standardized  UXO 
Technology  Demonstration  site  to  conduct  experiments  related  to  the  detection  of  deep  targets. 
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b.  Concentrating  activities  on  the  calibration  and  blind  grids.  Surveying  the  calibration 
grid  in  order  to  document  how  changes  in  the  dynamic  acquisition  parameters  (e.g.,  survey 
speed,  sample  rate,  and  base  frequency)  affect  the  quality  of  the  resulting  detection  maps.  G&G 
Sciences,  Inc.  observed  from  the  GT  for  the  reconfigured  calibration  lanes  that,  unlike  at  YPG, 
there  are  no  longer  targets  in  the  calibration  grid  that  truly  test  the  detection  limits  of  the  AOL 
system  (e.g.,  81-mm  mortars  at  >1  m  and  12-lb  shot  puts  at  >1  m).  Presumed  in  the  blind  grid, 
some  targets  have  been  seeded  at  depths  designed  to  test  the  detection  limits  of  the  technology. 
Repeated  surveys  over  the  blind  grid  are  planned  in  order  to  see  whether  there  is  a  particular  set 
of  survey  parameters  that  can  better  detect  deep  targets.  Keeping  in  mind  that  a  prioritized  target 
list  must  be  submitted  to  ATC  for  scoring,  a  set  of  static  data  over  the  blind  and  calibration  grids 
will  be  acquired.  Station  locations  will  also  be  acquired  with  an  RTK  GPS  system  with  the  base 
station  located  at  one  of  the  benchmark  locations  at  the  UXO  site.  As  alluded  to  above,  the 
acquisition  software  constantly  monitors  the  quality  of  the  GPS  solution  and  when  that  quality 
degrades  so  that  the  positions  are  not  RTK  quality,  a  visual  warning  appears  on  the  DAQ 
monitor.  RTK  quality  positions  with  accuracies  on  the  order  of  centimeters  are  essential  for  the 
high  resolution  dynamic  surveys  that  are  intended  to  be  conducted. 

c.  Dynamic  Survey.  Dynamic  surveys  over  both  the  calibration  grid  and  the  blind  test 
grid  will  be  conducted.  These  surveys  will  be  conducted  using  excitation  with  a  single 
transmitter  loop  at  1 -meter  lane  intervals.  All  or  part  of  these  surveys  may  be  repeated  using 
different  acquisition  parameters.  The  maps  that  are  compiled  from  these  data  will  be  used  for 
target  detection. 

d.  Calibration  Checks.  Proper  functioning  of  both  navigation  and  EM  data  acquisition 
will  be  assured  by  conducting  periodic  calibration  surveys  as  described  earlier.  These  surveys 
provide  a  check  of  the  three  critical  AOL2  subsystems,  navigation,  attitude,  and  EM  data 
acquisitions,  as  well  as  serving  as  a  means  to  sample  long  tenn  drift  of  the  instrument  response. 

e.  Static  Surveys.  Using  the  target  list  generated  from  the  dynamic  surveys  above,  each 
target  will  be  reacquired,  and  a  static  data  set  will  be  taken  that  will  consist  of  the  EMI  response 
from  all  three  transmitter  polarizations.  For  static  measurements,  DAQ  parameters  will  be 
changed  to  allow  the  acquisition  of  a  longer  time  transient  (e.g.,  T  =  0.3  s,  N  =  9  to  provide  us 
with  an  8.3  ms  transient  decay).  The  acquisition  stack-count  is  generally  set  so  that  a  data  point 
is  acquired  in  less  than  30  seconds.  Previous  experience  with  this  survey  mode  has  demonstrated 
that  it  is  capable  of  acquiring  up  to  200  targets  per  day.  As  with  the  dynamic  surveys,  a  repeat  of 
all  or  part  of  the  two  grids  using  different  acquisition  parameters  may  occur. 

f.  Calibration  Checks.  All  static  surveys  will  include  periodic  measurements  at  a 
background  site  and  over  a  calibration  target.  Furthermore,  the  intent  is  to  acquire  static 
background  points  at  random  locations  within  the  calibration  and  blind  grids  that  are  judged  to 
be  background  with  the  objective  of  detennining  whether  background  varies  with  position.  The 
frequency  of  the  calibration  checks  will  depend  on  the  drift  rates  that  are  observed  during 
surveys  over  the  calibration  grid.  At  a  minimum,  however,  these  calibration  checks  will  be  run 
two  times  daily  at  the  start  of  the  field  day  and  at  quitting  time. 
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2.1.6  Additional  Records 


The  following  record(s)  by  this  vendor  can  be  accessed  via  the  Internet  as  Microsoft  Word 
documents  at  www .uxotestsites .  or g . 
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2.2  APG  SITE  INFORMATION 


2.2.1  Location 


The  APG  Standardized  Test  Site  is  located  within  a  secured  range  area  of  the  Aberdeen 
Area.  The  Aberdeen  Area  of  APG  is  located  approximately  30  miles  northeast  of  Baltimore  at 
the  northern  end  of  the  Chesapeake  Bay.  The  Standardized  Test  Site  encompasses  17  acres  of 
upland  and  lowland  flats,  woods,  and  wetlands. 

2.2.2  Soil  Type 

According  to  the  soils  survey  conducted  for  the  entire  area  of  APG  in  1998,  the  test  site 
consists  primarily  of  Elkton  Series  type  soil  (ref  2).  The  Elkton  Series  consist  of  very  deep, 
slowly  penneable,  poorly  drained  soils.  These  soils  formed  in  silty  aeolin  sediments  and  the 
underlying  loamy  alluvial  and  marine  sediments.  They  are  on  upland  and  lowland  flats  and  in 
depressions  of  the  Mid-Atlantic  Coastal  Plain.  Slopes  range  from  0  to  2  percent. 

ERDC  conducted  a  site-specific  analysis  in  May  2002  (ref  3).  The  results  basically 
matched  the  soil  survey  mentioned  above.  Seventy  percent  of  the  samples  taken  were  classified 
as  silty  loam.  The  majority  (77  percent)  of  the  soil  samples  had  a  measured  water  content 
between  15  and  30  percent  with  the  water  content  decreasing  slightly  with  depth. 

For  more  details  concerning  the  soil  properties  at  the  APG  test  site,  go  to 
www.uxotestsites.org  on  the  Web  to  view  the  entire  soils  description  report. 

2.2.3  Test  Areas 


A  description  of  the  test  site  areas  at  APG  is  presented  in  Table  1.  A  test  site  layout  is 
shown  in  Figure  2. 
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TABLE  1.  TEST  SITE  AREAS 


Area 

Description 

Calibration  lanes 

Contains  14  standard  munitions  items  buried  in  six  positions,  with  representation 
of  clutter,  at  various  angles  and  depths  to  allow  demonstrators  to  calibrate  their 
equipment. 

Blind  grid 

Contains  400  grid  cells  in  a  0.5-acre  site.  The  center  of  each  grid  cell  contains 
either  munitions,  clutter,  or  nothing. 

Open  field 

A  10-acre  site  composed  of  generally  open  and  flat  terrain  with  minimal  clutter 
and  minor  navigational  obstacles.  Vegetation  height  varies  from  15  to  25  cm. 
This  area  is  subdivided  into  four  subareas  (legacy,  direct  fire,  indirect  fire,  and 
challenge). 

•  Open  field  (legacy) 

The  legacy  subarea  contains  the  same  wide  variety  of  randomly-placed  munitions 
that  were  present  in  the  open  field  prior  to  the  January  2008  general 
reconfiguration  of  the  site. 

•  Open  field  (direct  fire) 

The  direct  fire  subarea  contains  only  three  munition  types  that  could  be  typically 
found  at  an  impact  area  of  a  direct  fire  weapons  range.  Munitions  and  clutter  are 
placed  in  a  pattern  typical  for  these  munitions. 

•  Open  field  (indirect  fire) 

The  indirect  fire  subarea  contains  only  three  munition  types  that  could  be  typically 
found  at  an  impact  area  of  an  indirect  fire  weapons  range.  Munitions  and  clutter 
are  placed  in  a  pattern  typical  for  these  munitions. 

•  Open  field  (challenge) 

The  challenge  subarea  is  easily  reconfigurable  used  to  meet  the  specific  needs  and 
requirements  of  the  demonstrator  or  the  program  sponsor.  Any  results  from  this 
area  will  not  be  reported  in  the  standardized  scoring  record. 

Woods 

1.34-acre  area  consisting  of  cleared  woods  (tree  removal  with  only  stumps 
remaining),  partially  cleared  woods  (including  all  underbrush  and  fallen  trees), 
and  virgin  woods  (i.e.,  woods  in  natural  state  with  all  trees,  underbrush,  and 
fallen  trees  left  in  place). 

Moguls 

1.30-acre  area  consisting  of  two  areas  (the  rectangular  or  driving  portion  of  the 
course  and  the  triangular  section  with  more  difficult,  nondrivable  terrain).  A 
series  of  craters  (as  deep  as  0.91  m)  and  mounds  (as  high  as  0.91  m)  encompass 
this  section. 

2.2.4  STANDARD  AND  NONSTANDARD  INERT  MUNITIONS  TARGETS 

The  standard  and  nonstandard  munitions  items  emplaced  in  the  test  areas  are  presented  in 
Table  2.  Standardized  targets  are  members  of  a  set  of  specific  munitions  items  that  have 
identical  properties  to  all  other  items  in  the  set  (caliber,  configuration,  size,  weight,  aspect  ratio, 
material,  filler,  magnetic  remanence,  and  nomenclature).  Nonstandard  targets  are  inert 
munitions  items  having  properties  that  differ  from  those  in  the  set  of  standardized  items. 
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TABLE  2.  INERT  MUNITIONS  TARGETS 


Munition 

Calibration 

Open  Field 

Open  field 

Open  Field 

Item 

Type 

Lanes 

Blind  Grid 

Direct  Fire 

Indirect  Fire 

Legacy 

Moguls 

Woods 

20-mm  Projectile  M55 

S 

X 

X 

X 

X 

25-mm  Projectile  M794 

S 

X 

X 

X 

37-mm  Projectile  M47 

s 

X 

X 

X 

40-mm  Projectile  MKII  Bodies 

s 

X 

X 

X 

X 

BDU-28  Submunition 

s 

X 

X 

X 

X 

BLU-26  Submunition 

s 

X 

X 

X 

X 

M42  Submunition 

s 

X 

X 

X 

X 

57-mm  Projectile  APC  M86 

s 

X 

X 

X 

X 

60-mm  Mortar  M49A3 

s 

X 

X 

X 

2.75-in.  Rocket  M230 

s 

X 

X 

X 

X 

81 -mm  Mortar  M3  74 

s 

X 

X 

X 

X 

X 

X 

105-mm  HEAT  Rounds  M456 

s 

X 

X 

X 

105-mm  HEAT  Round  M490 

s 

X 

X 

X 

105-mm  Projectile  M60 

s 

X 

X 

X 

X 

X 

X 

155-mm  Projectile  M483A1 

s 

X 

X 

X 

X 

20-mm  Projectile  M55 

NS 

X 

X 

X 

20-mm  Projectile  M97 

NS 

X 

X 

X 

40-mm  Projectile  M813 

NS 

X 

X 

X 

60-mm  Mortar  (JPG) 

NS 

X 

X 

X 

60-mm  Mortar  M49 

NS 

X 

X 

X 

2.75-in.  Rocket  M230 

NS 

X 

X 

X 

2.75-in.  Rocket  XM229 

NS 

X 

X 

X 

81 -mm  Mortar  (JPG) 

NS 

X 

X 

X 

81 -mm  Mortar  M3  74 

NS 

X 

X 

X 

105-mm  Projectile  M60 

NS 

X 

X 

X 

155-mm  Projectile  M483A 

NS 

X 

X 

X 

S 

NS 

JPG 

HEAT 


Standard  munition. 
Nonstandard  munition. 
Jefferson  Proving  Ground, 
high-explosive  antitank. 


Figure  2.  Test  site  layout. 
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SECTION  3.  FIELD  DATA 


3.1  DATE  OF  FIELD  ACTIVITIES  (29  and  30  September,  3  and  4  October  2008) 

3.2  AREAS  TESTED/NUMBER  OF  HOURS 

Areas  tested  and  total  numbers  of  hours  operated  at  each  site  are  presented  in  Table  3. 


TABLE  3.  AREAS  TESTED  AND 
NUMBER  OF  HOURS 


Area 

Number  of  Hours 

Calibration  lanes 

13.00 

Blind  grid 

7.42 

Open  field 

0.00 

Woods 

0.00 

Mogul 

0.00 

Mine  grid 

1.25 

Note:  Table  3  represents  the  total  time  spent  in  each  area. 


3.3  TEST  CONDITIONS 
3.3.1  Weather  Conditions 

An  APG  weather  station  located  approximately  1  mile  west  of  the  test  site  was  used  to 
record  average  temperature  and  precipitation  on  a  half  hour  basis  for  each  day  of  operation.  The 
temperatures  presented  in  Table  4  represent  the  average  temperature  during  field  operations  from 
0700  to  1700  hours,  while  precipitation  data  represents  a  daily  total  amount  of  rainfall.  Hourly 
weather  logs  used  to  generate  this  summary  are  provided  in  Appendix  B. 


TABLE  4.  TEMPERATURE/PRECIPITATION  DATA  SUMMARY 


Date,  08 

Average  Temperature,  °F 

Total  Daily  Precipitation,  in. 

29  Sep 

70.8 

0.00 

30  Sep 

67.4 

0.33 

03  Oct 

64.4 

0.00 

04  Oct 

62.7 

0.00  i 

3.3.2  Field  Conditions 


G&G  Sciences,  Inc.  surveyed  the  grids  the  last  days  of  September  and  the  first  couple  days 
of  October.  The  temperature  was  seasonable.  Rain  did  fall  on  30  September,  but  it  did  not 
hinder  the  survey.  The  field  was  in  excellent  shape  during  the  survey. 
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3.3.3  Soil  Moisture 


Three  soil  probes  were  placed  at  various  locations  within  the  site  to  capture  soil  moisture 
data:  blind  grid,  calibration,  open  field,  and  wooded  areas.  Measurements  were  collected  in 
percent  moisture  and  were  taken  twice  daily  (morning  and  afternoon)  from  five  different  soil 
depths  (1  to  6  in.,  6  to  12  in.,  12  to  24  in.,  24  to  36  in.,  and  36  to  48  in.)  from  each  probe.  Soil 
moisture  logs  are  provided  in  Appendix  C. 

3.4  FIELD  ACTIVITIES 

3.4.1  Setup/Mobilization 

These  activities  included  initial  mobilization  and  daily  equipment  preparation  and 
breakdown.  A  three-person  crew  took  2  hours  and  55  minutes  to  perform  the  initial  setup  and 
mobilization.  There  were  3  hours  and  50  minutes  of  daily  equipment  preparation,  and  end  of  the 
day  equipment  breakdown  lasted  60  minutes. 

3.4.2  Calibration 


G&G  Sciences,  Inc.  spent  a  total  of  13  hours  in  the  calibration  lanes,  of  which  9  hours 
were  spent  collecting  data.  Two  other  calibration  activities  occurred  during  the  survey  of  the 
blind  grid,  totaling  15  minutes. 

3.4.3  Downtime  Occasions 


Occasions  of  downtime  are  grouped  into  five  categories:  equipment/data  checks  or 
equipment  maintenance,  equipment  failure  and  repair,  weather,  demonstration  site  issues,  or 
breaks/lunch.  All  downtime  is  included  for  the  purposes  of  calculating  labor  requirements 
(section  5)  except  for  downtime  due  to  demonstration  site  issues.  Demonstration  site  issues, 
while  noted  in  the  daily  log,  are  considered  nonchargeable  downtime  for  the  purposes  of 
calculating  labor  costs  and  are  not  discussed.  Breaks  and  lunches  are  discussed  in  this  section 
and  billed  to  the  total  site  survey  area. 

3.4.3. 1  Equipment/data  checks,  maintenance.  Equipment  data  checks  and  maintenance 
activities  accounted  for  10  minutes  of  site  usage  time.  These  activities  included  changing  out 
batteries  and  perfonning  routine  data  checks  to  ensure  the  data  were  being  properly 
recorded/collected.  G&G  Sciences,  Inc.  spent  no  time  for  breaks  and  lunches. 

3.4.3.2  Equipment  failure  or  repair.  1  hour  and  15  minutes  was  needed  to  resolve  equipment 
failures  that  occurred  while  surveying.  The  wooden  arm  broke  twice  on  the  AOL  system.  It  was 
repaired  quickly  and  without  incident  both  times. 

3.4.3.3  Weather.  No  weather  delays  occurred  during  the  survey. 
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3.4.4  Data  Collection 


TABLE  5.  TOTAL  TIME  G&G  SCIENCES,  INC. 
SPENT  PER  AREA 


AREA 

Time,  hr/min 

Blind  grid 

7  hours/25  minutes 

Open  field 

N/A 

Legacy 

N/A 

Direct  fire 

N/A 

Indirect  fire 

N/A 

Challenge 

N/A 

Wooded 

N/A 

Mine  Grid 

1  hour/15  minutes 

Moguls 

N/A 

Note:  Table  5  represents  the  total  time  spent  in  each  area  collecting  data. 


3.4.5  Demobilization 


The  G&G  Sciences,  Inc.  survey  crew  went  on  to  conduct  a  demonstration  of  the  blind  and 
mine  grids.  Therefore,  demobilization  did  not  occur  until  4  October  2008.  On  that  day,  it  took 
the  crew  55  minutes  to  break  down  and  pack  up  their  equipment. 

3.5  PROCESSING  TIME 

G&G  Sciences,  Inc.  submitted  the  raw  data  from  the  demonstration  activities  on  the  last 
day  of  the  demonstration,  as  required.  The  scoring  submittal  data  were  also  provided  within  the 
required  30-day  time  frame. 

3.6  DEMONSTRATOR’S  FIELD  PERSONNEL 

Thomas  S.  King 
Donald  D.  Snyder 
David  C.  George 

3.7  DEMONSTRATOR’S  FIELD  SURVEYING  METHOD 

G&G  Sciences,  Inc.  collected  the  data  in  a  linear  fashion.  They  used  line  spacing  of 
1/2  meter. 

3.8  SUMMARY  OF  DAILY  LOGS 

Daily  logs  capture  all  field  activities  during  this  demonstration  and  are  provided  in 
Appendix  D.  Activities  pertinent  to  this  specific  demonstration  are  indicated  in  highlighted  text. 
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(Page  24  Blank) 


SECTION  4.  TECHNICAL  PERFORMANCE  RESULTS 


4.1  ROC  CURVES  USING  ALL  MUNITIONS  CATEGORIES 

The  probability  of  detection  for  the  response  stage  (Pdres)  and  the  discrimination  stage 
(P,idlsc)  versus  their  respective  probability  of  clutter  detection  or  probability  of  false  positive 
within  each  area  are  shown  in  Figures  3  through  10.  The  probabilities  plotted  against  their 
respective  background  alarm  rate  within  each  area  are  shown  in  Figures  11  through  18.  Both 
figures  use  horizontal  lines  to  illustrate  the  perfonnance  of  the  demonstrator  at  two 
demonstrator-specified  points:  at  the  system  noise  level  for  the  response  stage,  representing  the 
point  below  which  targets  are  not  considered  detectable,  and  at  the  demonstrator’s  recommended 
threshold  level  for  the  discrimination  stage,  defining  the  subset  of  targets  the  demonstrator  would 
recommend  digging  based  on  discrimination. 


Figure  3.  AOL/towed  (EM)  blind  grid  probability  of  detection  for  response  and  discrimination 
stages  versus  their  respective  probability  of  false  positive. 
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Prob  of  False  Positive 


I  -  -  -  ■  Response 

- Discrimination 

-  -  -  ■  hbise  Level 
-  Threshold 


Figure  4.  AOL/towed  (MAG)  blind  grid  probability  of  detection  for  response  and  discrimination 
stages  versus  their  respective  probability  of  false  positive. 


Figure  5.  AOL/towed  mine  grid  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  probability  of  false  positive. 
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Not  covered 


Figure  6.  AOL/towed  open  field  (directfire)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  7.  AOL/towed  open  field  (indirectfire)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  8.  AOL/towed  open  field  (legacy)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  9.  AOL/towed  wooded  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  probability  of  false  positive. 

Not  covered 

Figure  10.  AOL/towed  mogul  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  probability  of  false  positive. 
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Figure  1 1 .  AOL/towed  (EM)  blind  grid  probability  of  detection  for  response  and  discrimination 
stages  versus  their  respective  probability  of  background  alarm. 


Figure  12.  AOL/towed  (MAG)  blind  grid  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  probability  of  background  alarm. 
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Figure  13.  AOL/towed  mine  grid  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  probability  of  background  alarm. 

Not  covered 

Figure  14.  AOL/towed  open  field  (direct  fire)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  background  alarm  rate. 

Not  covered 

Figure  15.  AOL/towed  open  field  (indirect  fire)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  background  alarm  rate. 


Not  covered 

Figure  16.  AOL/towed  open  field  (legacy)  probability  of  detection  for  response  and 
discrimination  stages  versus  their  respective  background  alarm  rate. 


Not  covered 

Figure  17.  AOL/towed  wooded  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  background  alarm  rate. 
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Not  covered 


Figure  18.  AOL/towed  mogul  probability  of  detection  for  response  and  discrimination  stages 
versus  their  respective  background  alarm  rate. 


4.2  PERFORMANCE  SUMMARIES 

Results  for  each  of  the  testing  areas  are  presented  in  Table  6  (for  labor  requirements,  see 
section  5).  The  response  stage  results  are  derived  from  the  list  of  anomalies  above  the 
demonstrator-provided  noise  level.  The  results  for  the  discrimination  stage  are  derived  from  the 
demonstrator’s  recommended  threshold  for  optimizing  munitions  related  cleanup  by  minimizing 
false  alarm  digs  and  maximizing  munitions  recovery.  The  lower  and  upper  90-percent 
confidence  limits  on  Pd,  Pcd,  and  Pfp  were  calculated  assuming  that  the  number  of  detections  and 
false  positives  are  binomially  distributed  random  variables. 


TABLE  6a.  BLIND  GRID  TEST  AREA  RESULTS  (EM) 


1  Response  Stage 

Discrimination  Stage 

Munitions11 

Scores 

Pj"x:  by  type 

All  Types 

105-mm 

81/60-mm 

37/25-mm 

All  Types 

105-mm 

81/60-mm 

37/25-mm 

1.00 

1.00 

0.98 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

1.00 

0.99 

0.96 

1.00 

1.00 

0.93 

1.00 

1.00 

0.93 

1.00 

0.97 

0.88 

|  By  Depthb  | 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

8D  to  12D 

1.00 

1.00 

1.00 

1.00 

0.89 

0.83 

1.00 

1.00  i 

Clutter 

Scores 

Pc i 

Pfe 

1  By  Mass  | 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

1.00 

1.00 

0.98 

1.00 

1.00 

1.00 

0.78 

0.73 

0.67 

0.52 

0.90 

1.00 

0  to  0.15  m 

1.00 

1.00 

1.00 

1.00 

0.71 

0.53 

0.89 

1.00 

0.15  to  03  m 

1.00 

1.00 

1.00 

1.00 

0.81 

0.40 

1.00 

1.00 

0.3  to  0.6  m 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Background  Alann  Rates 

Pbares:  0.23  |  Pbadisc:  0.05 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6b.  BLIND  GRID  TEST  AREA  RESULTS  (MAG) 


[  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  P,"':  by  type  I 

Pdd,sc:  by  type 

All  Types 

105-inm 

81/60-mm 

37/25-mm 

All  Types 

105-mm 

81/60-mm 

37/25-mm 

1.00 

0.99 

0.96 

1.00 

1.00 

0.93 

1.00 

0.97 

0.88 

1.00 

1.00 

0.93 

1.00 

0.99 

0.96 

1.00 

1.00 

0.93 

1.00 

0.97 

0.88 

1.00 

1.00 

0.93 

|  By  Depthb  | 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

0.97 

1.00 

1.00 

0.95 

0.97 

1.00 

1.00 

0.95 

8D  to  12D 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00  ! 

Clutter 

Scores 

P.d 

Pfe 

|  By  Mass  | 

By  Depthb 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

0.80 

0.75 

0.69 

0.66 

0.83 

0.90 

0.80 

0.75 

0.69 

0.66 

0.83 

0.90 

0  to  0.15  m 

0.77 

0.68 

0.87 

0.83 

0.77 

0.68 

0.87 

0.83 

0.15  to  03  m 

0.63 

0.40 

0.57 

1.00 

0.63 

0.40 

0.57 

1.00 

0.3  to  0.6  m 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

N/A 

Background  Alann  Rates 

Pbares:  0.11  |  Pbaa,sc:  0.11 

TABLE  6c.  MINE  GRID  TEST  AREA  RESULTS 


I  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

Pd"':  byt 

ype  1 

P/‘SC:  by  type 

All 

Types 

VS50 

TM62 

VS1.6 

M14 

All 

Types 

VS50 

TM62 

VS1.6 

M14 

0.71 

0.59 

0.45 

1.00 

1.00 

0.75 

1.00 

1.00 

0.63 

0.54 

0.25 

0.07 

0.54 

0.25 

0.07 

0.58 

0.45 

0.32 

0.85 

0.63 

0.35 

1.00 

1.00 

0.63 

0.41 

0.13 

0.01 

0.54 

0.25 

0.07 

Clutter 

Scores 

P.d 

Pfn 

By  Depthh 

All 

Depth 

Oto  15 

m 

0.15  to 
0.3  m 

0.3  to 

0.6  m 

>  0.6  m 

All 

Depth 

Oto  15 

m 

0.15  to 
0.3  m 

0.3  to 

0.6  m 

>0.6 

m 

All  Mass 

0.77 

0.68 

0.56 

0.55 

0.71 

0.75 

N/A 

0.66 

0.55 

0.44 

0.36 

0.65 

0.58 

N/A 

Background  Alann  Rates 

Pbar£S:  0.39  |  Pbadisc:  0.19 


aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6d.  OPEN  FIELD  DIRECT  FIRE  TEST  AREA  RESULTS 


Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

Pd**:  by  type 

PJ"'C:  by  type 

All  Types 

105-mm 

37-mm 

2  5- mm 

All  Types 

105-mm 

37-mm 

25-mm 

- 

-- 

- 

1  1 

1  1 

1 

1 

- 

- 

- 

- 

By  Density  | 

High 

- 

— 

- 

- 

- 

— 

— 

- 

Medium 

— 

— 

- 

- 

- 

— 

— 

- 

Low 

- 

- 

- 

- 

- 

- 

- 

- 

By  Depthb  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

- 

- 

- 

— 

— 

- 

8D  to  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

Pc, 

Pfe 

By  Mass  | 

By  Depthb 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alann  Rates 

BARres:  --  |  BARa,sc:  -- 

Groups 

Found 

— 

- 

Identified 

- 

- 

Coverage 

-- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6e.  OPEN  FIELD  INDIRECT  FIRE  TEST  AREA  RESULTS 


1  Response  Stage 

Discrimination  Stage 

Munitions3 

P/es:  by  type 

Pj‘sc:  by  type 

Scores 

All  Types 

105-nun 

37- mm 

|  2  5- mm 

All  Types 

105-mm 

37-mm 

25-mm 

-- 

- 

- 

1  1 

1  1 

1 

1 

- 

- 

- 

- 

|  By  Density  j 

High 

- 

- 

- 

- 

- 

- 

- 

- 

Medium 

— 

- 

— 

— 

- 

- 

— 

— 

Low 

— 

- 

— 

— 

- 

- 

— 

— 

|  By  Depthb  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

— 

— 

— 

— 

— 

— 

8D  to  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

Pc „ 

Pfe 

J  By  Mass  | 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

|  Background  Alarm  Rates  | 

1 

BARres:  - 

|  BARdisc:  - 

|  Groups  | 

Found 

— 

— 

Identified 

— 

— 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6f.  OPEN  FIELD  LEGACY  TEST  AREA  RESULTS 


Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

P"':  by  type 

Pdd,sc:  by  type 

All  Types 

105- mm 

37-mm 

2  5- mm 

All  Types 

105-mm 

37-mm 

25-mm 

-- 

- 

- 

- 

1  1 

1  1 

1 

1 

l  l 

l  l 

1 

1 

- 

i  i 

i  i 

By  Density 

High 

— 

— 

— 

— 

- 

- 

— 

— 

Medium 

— 

- 

— 

— 

- 

- 

— 

— 

Low 

- 

- 

- 

- 

- 

- 

- 

- 

By  Depthb 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

— 

— 

- 

- 

— 

— 

8D  to  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

PcU 

Pfe 

By  Mass 

By  Depthb 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

-- 

- 

- 

- 

1  1 

l  l 

j 

- 

- 

- 

0  to  0.15  m 

— 

- 

- 

- 

- 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alann  Rates 

BARres:  --  |  BARa,sc:  -- 

Groups 

Found 

— 

- 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6g.  WOODED  TEST  AREA  RESULTS 


j  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pd‘s:  by  type  \ 

Pf'sc:  by  type 

All  Types 

105- mm 

37-mm 

i  2  5- mm  j 

All  Types 

105-mm 

37-mm 

25-mm 

-- 

- 

- 

1  1 

1  1 

1 

1 

- 

- 

- 

- 

j  By  Density  | 

High 

— 

— 

— 

— 

- 

- 

— 

— 

Medium 

— 

- 

— 

— 

- 

- 

— 

— 

Low 

- 

- 

- 

- 

- 

- 

- 

- 

1  By  Depthb  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

— 

— 

- 

- 

— 

— 

8D  to  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

Pcd 

Pfe 

1  By  Mass  | 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

-- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

— 

- 

- 

- 

- 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alann  Rates 

BARres:  --  |  BARa,sc:  -- 

Groups 

Found 

— 

- 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 
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TABLE  6h.  MOGUL  TEST  AREA  RESULTS 


j  Response  Stage 

Discrimination  Stage 

Munitions3 

Scores 

1  Pd‘s:  by  type  | 

Pdd,sc:  by  type 

All  Types 

105- mm 

37-mm 

i  2  5- mm  j 

All  Types 

105-mm 

37-mm 

25-mm 

-- 

- 

- 

1  1 

1  1 

1 

1 

- 

- 

- 

- 

j  By  Density  | 

High 

— 

— 

— 

— 

- 

- 

— 

— 

Medium 

— 

- 

— 

— 

- 

- 

— 

— 

Low 

- 

- 

- 

- 

- 

- 

- 

- 

1  By  Depthb  | 

0  to  4D 

- 

- 

- 

- 

- 

- 

- 

- 

4D  to  8D 

— 

— 

— 

— 

- 

- 

— 

— 

8D  to  12D 

- 

- 

- 

- 

- 

- 

- 

- 

Clutter 

Scores 

PcU 

pfe 

1  By  Mass  | 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Mass 

0  to  0.25  kg 

>0.25  to 

1  kg 

>1  to  8  kg 

All  Depth 

-- 

- 

- 

- 

- 

- 

- 

- 

0  to  0.15  m 

— 

- 

- 

- 

- 

- 

- 

- 

0.15  to  03  m 

- 

- 

- 

- 

- 

- 

- 

- 

0.3  to  0.6  m 

- 

- 

- 

- 

- 

- 

- 

- 

Background  Alann  Rates 

BARres:  --  |  BARa,sc:  -- 

Groups 

Found 

— 

- 

Identified 

- 

- 

Coverage 

- 

- 

aThe  two  numbers  to  the  right  of  the  all  types  munitions  result  are  an  upper  and  lower  90-percent 
confidence  interval  for  an  assumed  binomial  distribution. 
bAll  depths  are  measured  to  the  center  of  the  object. 


4.3  EFFICIENCY,  REJECTION  RATES,  AND  TYPE  CLASSIFICATION 

Efficiency  and  rejection  rates  are  calculated  to  quantify  the  discrimination  ability  at 
specific  points  of  interest  on  the  ROC  curve:  (1)  at  the  point  where  no  decrease  in  Pd  is  suffered 
(i.e.,  the  efficiency  is  by  definition  equal  to  one)  and  (2)  at  the  operator  selected  threshold. 
These  values  are  presented  in  Tables  7a  through  7h. 
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TABLE  7a.  BLIND  GRID  EFFICIENCY  AND  REJECTION  RATES  (EM) 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

0.99 

0.28 

0.77 

With  No  Loss  of  Pd 

1.00 

0.00 

0.00 

TABLE  7b.  BLIND  GRID  EFFICIENCY  AND  REJECTION  RATES  (MAG) 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

1.00 

0.00 

0.00 

With  No  Loss  of  Pd 

1.00 

0.00 

0.00 

TABLE  7c.  MINE  GRID  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

0.77 

0.19 

0.50 

With  No  Loss  of  Pd 

1.00 

0.00 

0.00 

TABLE  7d.  OPEN  FIELD  (DIRECT)  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

- 

-- 

-- 

TABLE  7e.  OPEN  FIELD  (INDIRECT)  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

- 

- 

-- 
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TABLE  7f.  OPEN  FIELD  (LEGACY)  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

— 

— 

- 

TABLE  7g.  WOODED  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

- 

- 

- 

TABLE  7h.  MOGUL  EFFICIENCY  AND  REJECTION  RATES 


Efficiency  (E) 

False  Positive 
Rejection  Rate 

Background  Alarm 
Rejection  Rate 

At  Operating  Point 

— 

— 

— 

With  No  Loss  of  Pd 

— 

- 

- 

At  the  demonstrator’s  recommended  setting,  the  munitions  items  that  were  detected  and 
correctly  discriminated  were  further  scored  on  whether  their  correct  type  could  be  identified 
(tables  8a  through  8h).  Correct  type  examples  include  20-mm  projectile,  105-tnm  HEAT 
projectile,  and  2.75-inch  Rocket.  A  list  of  the  standard  type  declaration  required  for  each 
munitions  item  was  provided  to  demonstrators  prior  to  testing.  The  standard  types  for  the  three 
example  items  are  20-inmP,  105H,  and  2.75-inch. 
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TABLE  8a.  BLIND  GRID  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS  (EM) 


Size 

Percentage  Correct 

25mm 

— 

37mm 

— 

60mm 

— 

81mm 

— 

105mm 

— 

105  artillery 

— 

Overall 

-- 

Note:  The  demonstrator  did  not  attempt  to  provide  type  classification  (if  applicable). 


TABLE  8b.  BLIND  GRID  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS  (MAG) 


Size 

Percentage  Correct 

25mm 

— 

37mm 

— 

60mm 

— 

81mm 

— 

105mm 

— 

105  artillery 

— 

Overall 

- 

Note:  The  demonstrator  did  not  attempt  to  provide  type  classification  (if  applicable). 
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TABLE  8c.  MINE  GRID  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS 


Size 

Percentage  Correct 

VS50 

— 

TM62 

— 

VS1.6 

— 

M14 

— 

Overall 

-- 

Note:  The  demonstrator  did  not  attempt  to  provide  type  classification  (if  applicable). 


TABLE  8d.  OPEN  FIELD  DIRECT  FIRE 
CORRECT  TYPE  CLASSIFICATION 
OF  TARGETS  CORRECTLY 
DISCRIMINATED  AS 
MUNITIONS 


Size 

Percentage  Correct 

60mm 

— 

81mm 

— 

105mm 

— 

Overall 

-- 

TABLE  8e.  OPEN  FIELD  INDIRECT  FIRE 
CORRECT  TYPE  CLASSIFICATION 
OF  TARGETS  CORRECTLY 
DISCRIMINATED  AS 
MUNITIONS 


Size 

Percentage  Correct 

25 -mm 

— 

37-mm 

— 

105-mm 

— 

Overall 

- 
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TABLE  8f.  OPEN  FIELD  LEGACY  CORRECT 
TYPE  CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 

TABLE  8g.  WOODED  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 

TABLE  8h.  MOGUL  CORRECT  TYPE 
CLASSIFICATION  OF  TARGETS 
CORRECTLY  DISCRIMINATED 
AS  MUNITIONS 


Size 

Percentage  Correct 

Small 

— 

Medium 

— 

Large 

— 

Overall 

-- 

4.4  LOCATION  ACCURACY 

The  mean  location  error  and  standard  deviations  are  presented  in  Tables  9a  through  9h. 
These  calculations  are  based  on  average  missed  distance  for  munitions  correctly  identified  during 
the  response  stage.  Depths  are  measured  from  the  center  of  the  munitions  to  the  surface.  For  the 
blind  grid,  only  depth  errors  are  calculated  because  (X,  Y)  positions  are  known  to  be  the  centers 
of  the  grid  square. 
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TABLE  9a.  BLIND  GRID  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION  (EM) 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

Depth 

N/A 

N/A 

TABLE  9b.  BLIND  GRID  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION  (MAG) 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

N/A 

N/A 

TABLE  9c.  MINE  GRID  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

- 

— 

N/A 

N/A 

TABLE  9d.  OPEN  FIELD  DIRECT  FIRE  MEAN 
LOCATION  ERROR  AND 
STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

Depth 

- 

-- 
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TABLE  9e.  OPEN  FIELD  INDIRECT  FIRE  MEAN  LOCATION 
ERROR  AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

Depth 

- 

-- 

TABLE  9f.  OPEN  FIELD  LEGACY  MEAN  LOCATION 
ERROR  AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

— 

Depth 

- 

- 

TABLE  9g.  WOODED  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

— 

- 

Depth 

- 

-- 

TABLE  9h.  MOGUL  MEAN  LOCATION  ERROR 
AND  STANDARD  DEVIATION 


Mean 

Standard  Deviation 

Northing 

— 

— 

Easting 

- 

- 

Depth 

- 

-- 
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SECTION  5.  ON-SITE  LABOR  REQUIREMENTS 


A  standardized  estimate  for  labor  associated  with  this  effort  was  calculated  as  follows:  the 
first  person  at  the  test  site  was  designated  supervisor,  the  second  person  was  designated  data 
analyst,  and  the  third  and  following  personnel  were  considered  field  support. 

Government  representatives  monitored  on-site  activity.  All  on-site  activities  were 
grouped  into  one  of  ten  categories:  initial  setup/mobilization,  daily  setup/stop,  calibration, 
collecting  data,  downtime  due  to  break/lunch,  downtime  due  to  equipment  failure,  downtime  due 
to  equipment/data  checks  or  maintenance,  downtime  due  to  weather,  downtime  due  to 
demonstration  site  issue,  or  demobilization.  The  daily  activity  log  is  provided  in  Appendix  D.  A 
summary  of  field  activities  is  provided  in  Section  3.4. 

The  standardized  estimate  of  the  labor  needed  to  perform  the  field  activities  is  presented  in 
Table  10.  Note  that  calibration  time  includes  time  spent  in  the  calibration  lanes  as  well  as  field 
calibrations.  Site  survey  includes  daily  setup/stop  time,  collecting  data,  breaks/lunch,  downtime 
due  to  equipment/data  checks  or  maintenance,  downtime  due  to  failure,  and  downtime  due  to 
weather. 


TABLE  10.  ON-SITE  LABOR  REQUIREMENTS 


No.  of  People 

Hours 

Initial  setup 

Supervisor 

1 

2.92 

Data  analyst 

1 

2.92 

Field  support 

1 

2.92 

Subtotal 

Calibration  site  survey 

Supervisor 

1 

13.25 

Data  analyst 

1 

13.25 

Field  support 

1 

13.25 

Subtotal 

Blind  grid  site  survey 

Supervisor 

1 

7.42 

Data  analyst 

1 

7.42 

Field  support 

1 

7.42 

Subtotal 

See  notes  at  end  of  table. 
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TABLE  10.  (CONT’D) 


No.  of  People 

Hours 

Open  field  site  survey 

Supervisor 

0 

0.00 

Data  analyst 

0 

0.00 

Field  support 

0 

0.00 

Subtotal 

0 

0.00 

Wooded  site  survey 

Supervisor 

0 

0.00 

Data  analyst 

0 

0.00 

Field  support 

0 

0.00 

Subtotal 

0 

0.00 

Mogul  site  survey 

Supervisor 

0 

0.00 

Data  analyst 

0 

0.00 

Field  support 

0 

0.00 

Subtotal 

0 

0.00 

Mine  grid  site  survey 

Supervisor 

1 

1.25 

Data  analyst 

1 

1.25 

Field  support 

1 

1.25 

Subtotal 

Demobilization 

Supervisor 

1 

0.92 

Data  analyst 

1 

0.92 

Field  support 

1 

0.92 

Subtotal 

Notes:  Calibration  time  includes  time  spent  in  the  calibration  lanes  as  well  as  calibration 
before  each  data  run. 

Site  survey  time  includes  daily  setup/stop  time,  collecting  data,  breaks/lunch,  downtime 
due  to  system  maintenance,  failure,  and  weather. 
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SECTION  6.  APPENDIXES 


APPENDIX  A.  TERMS  AND  DEFINITIONS 

GENERAL  DEFINITIONS 

Anomaly:  Location  of  a  system  response  deemed  to  warrant  further  investigation  by  the 
demonstrator  for  consideration  as  an  emplaced  munitions  item. 

Detection:  An  anomaly  location  that  is  within  Rhai0  of  an  emplaced  munitions  item. 

Military  Munitions  (MM):  Specific  categories  of  MM  that  may  pose  unique  explosive  safety 
risks,  including  UXO  as  defined  in  10  USC  101(e)(5),  DMM  as  defined  in  10  USC  2710(e)(2) 
and/or  munitions  constituents  (e.g.  TNT,  RDX)  as  defined  in  10  USC  2710(e)(3)  that  are  present 
in  high  enough  concentrations  to  pose  an  explosive  hazard. 

Emplaced  Munitions:  A  munitions  item  buried  by  the  government  at  a  specified  location  in  the 
test  site. 

Emplaced  Clutter:  A  clutter  item  (i.e.,  nonmunitions  item)  buried  by  the  government  at  a 
specified  location  in  the  test  site. 

Rhai0:  A  predetermined  radius  about  an  emplaced  item  (clutter  or  munitions)  within  which  an 
anomaly  identified  by  the  demonstrator  as  being  of  interest  is  considered  to  be  a  detection  of  that 
item.  For  the  purpose  of  this  program,  a  circular  halo  0.5  meters  in  radius  is  placed  around  the 
center  of  the  object  for  all  clutter  and  munitions  items. 

Small  Munitions:  Caliber  of  munitions  less  than  or  equal  to  40  mm  (includes  20-mm  projectile, 
25-inm  projectile,  37-inm  projectile,  40-nnn  projectile,  submunitions  BLU-26,  BLU-63,  and 
M42). 

Medium  Munitions:  Caliber  of  munitions  greater  than  40  mm  and  less  than  or  equal  to  8 1  nun 
(includes  57-inm  projectile,  60-inm  mortar,  2.75-inch  rocket,  and  81-min  mortar). 

Large  Munitions:  Caliber  of  munitions  greater  than  81  mm  (includes  105-mrn  HEAT,  105-mrn 
projectile,  and  1 5 5 -mm  projectile). 

Group:  Two  or  more  adjacent  GT  items  with  overlapping  halos. 

GT:  Ground  truth 

Response  Stage  Noise  Level:  The  level  that  represents  the  signal  level  below  which  anomalies 
are  not  considered  detectable.  Demonstrators  are  required  to  provide  the  recommended  noise 
level  for  the  blind  grid  test  area. 
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Discrimination  Stage  Threshold:  The  demonstrator-selected  threshold  level  that  is  expected  to 
provide  optimum  performance  of  the  system  by  retaining  all  detectable  munitions  and  rejecting 
the  maximum  amount  of  clutter.  This  level  defines  the  subset  of  anomalies  the  demonstrator 
would  recommend  digging  based  on  discrimination. 

Binomially  Distributed  Random  Variable:  A  random  variable  of  the  type  which  has  only  two 
possible  outcomes,  say  success  and  failure,  is  repeated  for  n  independent  trials  with  the 
probability  p  of  success  and  the  probability  1-p  of  failure  being  the  same  for  each  trial.  The 
number  of  successes  x  observed  in  the  n  trials  is  an  estimate  of  p  and  is  considered  to  be  a 
binomially  distributed  random  variable. 

RESPONSE  AND  DISCRIMINATION  STAGE  DATA 

The  scoring  of  the  demonstrator’s  performance  is  conducted  in  two  stages:  response  stage 
and  discrimination  stage.  For  both  stages,  the  probability  of  detection  (Pd)  and  the  false  alarms 
are  reported  as  receiver-operating  characteristic  (ROC)  curves.  False  alarms  are  divided  into 
those  anomalies  that  correspond  to  emplaced  clutter  items,  measuring  the  probability  of  clutter 
detection  (Pcd)  or  probability  of  false  positive  (Pfp).  Those  that  do  not  correspond  to  any  known 
item  are  termed  background  alanns. 

The  response  stage  is  a  measure  of  whether  the  sensor  can  detect  an  object  of  interest.  For 
a  channel  instrument,  this  value  should  be  closely  related  to  the  amplitude  of  the  signal.  The 
demonstrator  must  report  the  response  level  (threshold)  below  which  target  responses  are 
deemed  insufficient  to  warrant  further  investigation.  At  this  stage,  minimal  processing  may  be 
done.  This  includes  filtering  long-  and  short-scale  variations,  bias  removal,  and  scaling.  This 
processing  should  be  detailed  in  the  data  submission. 

For  a  multichannel  instrument,  the  demonstrator  must  construct  a  quantity  analogous  to 
amplitude.  The  demonstrator  should  consider  what  combination  of  channels  provides  the  best 
test  for  detecting  any  object  that  the  sensor  can  detect.  The  average  amplitude  across  a  set  of 
channels  is  an  example  of  an  acceptable  response  stage  quantity.  Other  methods  may  be  more 
appropriate  for  a  given  sensor.  Again,  minimal  processing  can  be  done,  and  the  demonstrator 
should  explain  how  this  quantity  was  constructed  in  their  data  submission. 

The  discrimination  stage  evaluates  the  demonstrator’s  ability  to  correctly  identify 
munitions  as  such,  and  to  reject  clutter.  For  the  same  locations  as  in  the  response  stage  anomaly 
list,  the  discrimination  stage  list  contains  the  output  of  the  algorithms  applied  in  the 
discrimination-stage  processing.  This  list  is  prioritized  based  on  the  demonstrator’s 
determination  that  an  anomaly  location  is  likely  to  contain  munitions.  Thus,  higher  output  values 
are  indicative  of  higher  confidence  that  a  munitions  item  is  present  at  the  specified  location.  For 
electronic  signal  processing,  priority  ranking  is  based  on  algorithm  output.  For  other  systems, 
priority  ranking  is  based  on  human  judgment.  The  demonstrator  also  selects  the  threshold  that 
the  demonstrator  believes  will  provide  optimum  system  performance,  (i.e.,  that  retains  all  the 
detected  munitions  and  rejects  the  maximum  amount  of  clutter). 
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Note:  The  two  lists  provided  by  the  demonstrator  contain  identical  numbers  of  potential  target 
locations.  They  differ  only  in  the  priority  ranking  of  the  declarations. 

GROUP  SCORING  FACTORS 

Based  on  configuration  of  the  GT  at  the  standardized  sites  and  the  defined  scoring 
methodology,  there  exists  munitions  groups  defined  as  having  overlapping  halos.  In  these  cases, 
the  following  scoring  logic  is  implemented  (fig.  A-l  through  A-9): 

a.  Overall  site  scores  (i.e.,  Pd)  will  consider  only  isolated  munitions  and  clutter  items. 

b.  GT  items  that  have  overlapping  halos  (both  munitions  and  clutter)  will  form  a  group 
and  groups  may  form  chains. 

c.  Groups  will  have  a  complex  halos  composed  of  all  the  composite  halos  of  all  its  GT 

items. 

d.  Groups  will  have  three  scoring  factors:  groups  found  groups  identified  and  group 
coverage.  Scores  will  be  based  on  1 : 1  matches  of  anomalies  and  GT. 

(1)  Groups  Found  (Found):  the  number  of  groups  that  have  one  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  detecting  a 
group  if  any  item  within  the  group  is  matched  to  an  anomaly  in  their  list. 

(2)  Groups  Identified  (ID):  the  number  of  groups  that  have  two  or  more  GT  items 
matched  divided  by  the  total  number  of  groups.  Demonstrators  will  be  credited  with  identifying 
that  a  group  is  present  if  multiple  items  within  the  composite  halo  are  matched  to  anomalies  in 
their  list. 

(3)  Group  Coverage  (Coverage):  the  number  of  GT  items  matched  within  groups  divided 
by  the  total  number  of  GT  items  within  groups.  This  metric  measures  the  demonstrator  accuracy 
in  determining  the  number  of  anomalies  within  a  group.  If  five  items  are  present  and  only  two 
anomalies  are  matched,  the  demonstrator  will  score  0.4.  If  all  five  are  matched  the  demonstrator 
will  score  1.0. 

e.  Location  error  will  not  be  reported  for  groups. 
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f.  Demonstrators  will  not  be  asked  to  call  out  groups  in  their  scoring  submissions.  If 
multiple  anomalies  are  indicated  in  a  small  area,  the  demonstrator  will  report  all  individual 
anomalies. 

g.  Excess  alarms  within  a  halo  will  be  disregarded. 


A- 1 .  Example  of  detected  item. 
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A-3.  Example  of  group  identified  (ID). 


A-4.  Example  of  excess  alarms  disregarded. 
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A-6.  Example  of  group  (1/4  =  0.25). 
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A-7.  Example  of  group  (2/4  =  0.5). 
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RESPONSE  STAGE  DEFINITIONS 

Response  Stage  Probability  of  Detection  (Pdres):  Pdres  =  (No.  of  response-stage  detections)/ 
(No.  of  emplaced  munitions  in  the  test  site). 

Response  Stage  Clutter  Detection  (cdies):  An  anomaly  location  that  is  within  Rhai0  of  an 
emplaced  clutter  item. 

Response  Stage  Probability  of  Clutter  Detection  (Pcdres):  Pcdres  =  (No.  of  response-stage  clutter 
detections)/(No.  of  emplaced  clutter  items). 

Response  Stage  Background  Alarm  (bares):  An  anomaly  in  a  blind  grid  cell  that  contains  neither 
emplaced  munitions  nor  an  emplaced  clutter  item.  An  anomaly  location  in  the  open  field  or 
scenarios  that  is  outside  Rhaio  of  any  emplaced  munitions  or  emplaced  clutter  item. 

Response  Stage  Probability  of  Background  Alarm  (Pbares):  Blind  grid  only:  Pbares  =  (No.  of 
response-stage  background  alarms)/(No.  of  empty  grid  locations). 

Response  Stage  Background  Alarm  Rate  (BARres):  Open  field  any  challenge  area  (including  the 
direct  and  indirect  firing  sub  areas)  only:  BARies  =  (No.  of  response-stage  background 
alarms)/(arbitrary  constant). 

Note  that  the  quantities  Pdres,  Pcdres,  Pbares,  and  BARres  are  functions  of  tres,  the  threshold 
applied  to  the  response-stage  signal  strength.  These  quantities  can  therefore  be  written  as 

Pdres(tres),  Pcdres(tres),  Pbares(tres),  and  BARres(tres). 
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DISCRIMINATION  STAGE  DEFINITIONS 


Discrimination:  The  application  of  a  signal  processing  algorithm  or  human  judgment  to  sensor 
data  to  discriminate  munitions  from  clutter.  Discrimination  should  identify  anomalies  that  the 
demonstrator  has  high  confidence  correspond  to  munitions,  as  well  as  those  that  the  demonstrator 
has  high  confidence  correspond  to  nonmunitions  or  background  returns.  The  former  should  be 
ranked  with  highest  priority  and  the  latter  with  lowest. 

Discrimination  Stage  Probability  of  Detection  (Pddlsc):  Pddlsc  =  (No.  of  discrimination-stage 
detections)/(No.  of  emplaced  munitions  in  the  test  site). 

Discrimination  Stage  False  Positive  (fpdlsc):  An  anomaly  location  that  is  within  Rhaio  of  an 
emplaced  clutter  item. 

Discrimination  Stage  Probability  of  False  Positive  (Pfpdlsc):  Pfpdlsc  =  (No.  of  discrimination  stage 
false  positives)/(No.  of  emplaced  clutter  items). 

Discrimination  Stage  Background  Alarm  (badlsc):  An  anomaly  in  a  blind  grid  cell  that  contains 
neither  emplaced  munitions  nor  an  emplaced  clutter  item.  An  anomaly  location  in  the  open  field 
or  scenarios  that  is  outside  Rhaio  of  any  emplaced  munitions  or  emplaced  clutter  item. 

Discrimination  Stage  Probability  of  Background  Alarm  (Pbadlsc):  Pbadlsc  =  (No.  of  discrimination- 
stage  background  alarms)/(No.  of  empty  grid  locations). 

Discrimination  Stage  Background  Alarm  Rate  (BARdlsc):  BARdlsc  =  (No.  of  discrimination-stage 
background  alarms)/(arbitrary  constant). 

Note  that  the  quantities  Pddlsc,  Pfpdisc,  Pbadlsc,  and  BARdlsc  are  functions  of  tdlsc,  the  threshold 
applied  to  the  discrimination-stage  signal  strength.  These  quantities  can  therefore  be  written  as 
Pdd,sc(f,lsc),  Pfpdlsc(tdisc),  Pbadisc(tdisc),  and  BARdisc(tdisc). 
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RECEIVER-OPERATING  CHARACERISTIC  (ROC)  CURVES 


ROC  curves  at  both  the  response  and  discrimination  stages  can  be  constructed  based  on  the 
above  definitions.  The  ROC  curves  plot  the  relationship  between  Pd  versus  Pcd  or  Pfp  and  Pd 
versus  BAR  or  Pba  as  the  threshold  applied  to  the  signal  strength  is  varied  from  its  minimum 
(tmin)  to  its  maximum  (tmax)  value. 1  Pd  versus  Pfp  and  Pd  versus  BAR  being  combined  into  ROC 
curves  is  shown  in  Figure  A- 10.  Note  that  the  “res”  and  “disc”  superscripts  have  been 
suppressed  from  all  the  variables  for  clarity. 


Figure  A- 10.  ROC  curves  for  open  field  testing.  Each  curve  applies  to  both  the  response  and 
discrimination  stages. 

METRICS  TO  CHARACTERIZE  THE  DISCRIMINATION  STAGE 

The  demonstrator  is  also  scored  on  efficiency  and  rejection  ratio,  which  measure  the 
effectiveness  of  the  discrimination  stage  processing.  The  goal  of  discrimination  is  to  retain  the 
greatest  number  of  munitions  detections  from  the  anomaly  list  while  rejecting  the  maximum 
number  of  anomalies  arising  from  nonmunitions  items.  The  efficiency  measures  the  fraction  of 
detected  munitions  retained  by  the  discrimination,  while  the  rejection  ratio  measures  the  fraction 
of  false  alarms  rejected.  Both  measures  are  defined  relative  to  the  entire  response  list,  i.e.,  the 
maximum  munitions  detectable  by  the  sensor  and  its  accompanying  clutter  detection  rate/false 
positive  rate  or  background  alarm  rate. 

'Strictly  speaking,  ROC  curves  plot  the  Pd  versus  Pba  over  a  predetermined  and  fixed  number  of 
detection  opportunities  (some  of  the  opportunities  are  located  over  munitions  and  others  are 
located  over  clutter  or  blank  spots).  In  an  open  field  scenario,  each  system  suppresses  its  signal 
strength  reports  until  some  bare-minimum  signal  response  is  received  by  the  system. 
Consequently,  the  open  field  ROC  curves  do  not  have  information  from  low  signal-output 
locations,  and,  furthermore,  different  contractors  report  their  signals  over  a  different  set  of 
locations  on  the  ground.  These  ROC  curves  are  thus  not  true  to  the  strict  definition  of  ROC 
curves  as  defined  in  textbooks  on  detection  theory.  Note,  however,  that  the  ROC  curves 
obtained  in  the  blind  grid  test  sites  are  true  ROC  curves. 
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Efficiency  (E):  E  =  Pddlsc(tdlsc)/Pdres(tminres):  Measures  (at  a  threshold  of  interest)  the  degree 
to  which  the  maximum  theoretical  detection  performance  of  the  sensor  system  (as  determined  by 
the  response  stage  tmin)  is  preserved  after  application  of  discrimination  techniques.  Efficiency  is 
a  number  between  0  and  1 .  An  efficiency  of  1  implies  that  all  of  the  munitions  initially  detected 
in  the  response  stage  were  retained  at  the  specified  threshold  in  the  discrimination  stage,  tdlsc. 

False  Positive  Rejection  Rate  (Rfp):  Rfp  =  1  -  [Pfpdlsc(tdlsc)/PCdres(Wres)]:  Measures  (at  a 
threshold  of  interest)  the  degree  to  which  the  sensor  system's  false  positive  performance  is 
improved  over  the  maximum  false  positive  performance  (as  determined  by  the  response  stage 
tmin).  The  rejection  rate  is  a  number  between  0  and  1.  A  rejection  rate  of  1  implies  that  all 
emplaced  clutter  initially  detected  in  the  response  stage  were  correctly  rejected  at  the  specified 
threshold  in  the  discrimination  stage. 

Background  Alann  Rejection  Rate  (Rba): 

Blind  grid:  Rba  =  1  -  [Pbadlsc(tdlsc)/Pbares(tminres)]. 

Open  field:  Rba  =  1  -  [BARdlsc(tdlsc)/BARres(tminres)]). 

Measures  the  degree  to  which  the  discrimination  stage  correctly  rejects  background  alarms 
initially  detected  in  the  response  stage.  The  rejection  rate  is  a  number  between  0  and  1.  A 
rejection  rate  of  1  implies  that  all  background  alarms  initially  detected  in  the  response  stage  were 
rejected  at  the  specified  threshold  in  the  discrimination  stage. 

CHI-SQUARE  COMPARISON 

The  Chi-square  test  for  differences  in  probabilities  (or  2  by  2  contingency  table)  is  used  to 
analyze  two  samples  drawn  from  two  different  populations  to  see  if  both  populations  have  the 
same  or  different  proportions  of  elements  in  a  certain  category.  More  specifically,  two  random 
samples  are  drawn,  one  from  each  population,  to  test  the  null  hypothesis  that  the  probability  of 
event  A  (some  specified  event)  is  the  same  for  both  populations  (ref  3). 

The  test  statistic  of  the  2  by  2  contingency  table  is  the  Chi-square  distribution  with  one 
degree  of  freedom.  When  an  association  between  a  more  challenging  terrain  feature  and 
relatively  degraded  perfonnance  is  sought,  a  one-sided  test  is  performed.  A  two-sided  2  by  2 
contingency  table  is  used  in  the  Standardized  UXO  Technology  Demonstration  Site  Program  to 
compare  perfonnance  between  any  two  areas  or  subareas  when  the  direction  of  degradation 
cannot  be  predetermined. 

For  a  one-sided  test,  a  significance  level  of  0.05  is  used  to  set  the  critical  decision  limit.  It 
is  a  critical  decision  limit  because  if  the  test  statistic  calculated  from  the  data  exceeds  this  value, 
then  the  lower  proportion  tested  will  be  considered  significantly  less  than  the  greater  one 
(degraded).  If  the  test  statistic  calculated  from  the  data  is  less  than  this  value,  then  no 
degradation  can  be  said  to  exist  because  of  the  terrain  feature  introduced. 
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For  a  two-sided  test,  a  significance  level  of  0.10  is  used  to  allow  .05  on  either  side  of  the 
decision.  It  is  a  critical  decision  limit  because  if  the  test  statistic  calculated  from  the  data 
exceeds  this  value,  then  the  two  proportions  tested  will  be  considered  significantly  different.  If 
the  test  statistic  calculated  from  the  data  is  less  than  this  value,  then  the  two  proportions  tested 
will  be  considered  not  significantly  different. 

An  exception  must  be  applied  when  either  a  0  or  100  percent  success  rate  occurs  in  the 
sample  data.  The  Chi-square  test  cannot  be  used  in  these  instances.  Instead,  Fischer’s  test  is 
used,  and  the  critical  decision  limit  for  one-sided  tests  is  the  chosen  significance  level,  which  in 
this  case  is  0.05.  With  Fischer’s  test,  if  the  test  statistic  is  less  than  the  critical  value,  then  the 
proportions  are  considered  to  be  significantly  different. 

An  example  follows  that  illustrates  Standardized  UXO  Technology  Demonstration  Site 
blind  grid  results  compared  to  those  from  the  open  field  legacy.  It  should  be  noted  that  a 
significant  result  does  not  prove  a  cause-and-effect  relationship  exists  between  the  two 
populations  of  interest;  however,  it  does  serve  as  a  tool  to  indicate  that  one  data  set  has 
experienced  a  degradation  or  change  in  system  perfonnance  at  a  large  enough  level  than  can  be 
accounted  for  merely  by  chance  or  random  variation.  Note  also  that  a  result  that  is  not 
significant  indicates  that  there  is  not  enough  evidence  to  declare  that  anything  more  than  chance 
or  random  variation  within  the  same  population  is  at  work  between  the  two  data  sets  being 
compared. 

Demonstrator  X  achieves  the  following  overall  results  after  surveying  the  blind  grid  and 
open  field  (legacy)  using  the  same  system  (results  indicate  the  number  of  munitions  detected 
divided  by  the  number  of  munitions  emplaced): 

Blind  grid  Open  field 

Pdres  100/100  =  1.0  8/10  =  .80 


P/es:  BLIND  GRID  versus  OPEN  FIELD  (legacy).  Using  the  example  data  above  to 
compare  probabilities  of  detection  in  the  response  stage,  all  100  munitions  out  of  100  emplaced 
munitions  items  were  detected  in  the  blind  grid  while  8  munitions  out  of  10  emplaced  were 
detected  in  the  open  field.  Fischer’s  test  must  be  used  since  a  100  percent  success  rate  occurs  in 
the  data.  Fischer’s  test  uses  the  four  input  values  to  calculate  a  test  statistic  of  0.0075  that  is 
compared  against  the  critical  value  of  0.05.  Since  the  test  statistic  is  less  than  the  critical  value, 
the  smaller  response  stage  detection  rate  (0.80)  is  considered  to  be  significantly  less  at  the  0.05 
level  of  significance.  While  a  significant  result  does  not  prove  a  cause-and-effect  relationship 
exists  between  the  change  in  survey  area  and  degradation  in  performance,  it  does  indicate  that 
the  detection  ability  of  demonstrator  X’s  system  seems  to  have  been  degraded  in  the  open  field 
relative  to  results  from  the  blind  grid  using  the  same  system.  This  is  an  example  of  a  one-sided 
Chi- squared  test. 
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APPENDIX  B.  DAILY  WEATHER  LOGS 


Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

15  Sep 

0100 

81.1 

0.00  | 

0200 

82.0 

0.00 

0300 

80.8 

0.00 

0400 

79.5 

0.00 

0500 

77.5 

0.00 

0600 

77.4 

0.00 

0700 

76.6 

0.00  ! 

0800 

77.7 

0.00 

0900 

79.3 

0.00 

1000 

80.4 

0.00 

1100 

81.1 

0.00 

1200 

81.3 

0.00 

1300 

82.2 

0.00 

1400 

82.2 

0.00 

1500 

82.6 

0.00 

1600 

82.6 

0.00 

1700 

81.0 

0.00 

1800 

77.9 

0.00 

1900 

74.5 

0.00 

2000 

72.3 

0.00 

2100 

72.3 

0.00 

2200 

72.1 

0.00 

2300 

71.6 

0.00 

2359 

70.7 

0.00 

16  Sep 

0100 

69.1 

0.00 

0200 

67.3 

0.00 

0300 

65.5 

0.00 

0400 

63.7 

0.00 

0500 

63.0 

0.00  j 

0600 

62.6 

0.00 

0700 

62.8 

o.oo  : 

0800 

63.9 

0.00 

0900 

65.3 

0.00 

1000 

66.2 

0.00 

1100 

67.8 

0.00 

1200 

69.1 

0.00 

1300 

70.3 

0.00 

1400 

72.3 

0.00 

1500 

72.5 

0.00 

1600 

71.4 

0.00 

1700 

70.7 

0.00 

1800 

69.1 

0.00 

1900 

66.2 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

16  Sep 

2000 

62.6 

0.00 

2100 

60.3 

0.00 

2200 

58.5 

0.00 

2300 

57.2 

0.00 

2359 

56.3 

0.00 

17  Sep 

0100 

55.4 

0.00 

0200 

55.9 

0.00 

0300 

56.8 

0.00 

0400 

54.7 

0.00 

0500 

54.0 

0.00 

0600 

53.8 

0.00 

0700 

56.8 

0.00 

0800 

63.1 

0.00 

0900 

64.8 

0.00 

1000 

66.9 

0.00 

1100 

68.2 

0.00 

1200 

70.7 

0.00 

1300 

72.0 

0.00 

1400 

73.9 

0.00 

1500 

75.0 

0.00 

1600 

75.7 

0.00 

1700 

74.7 

0.00 

1800 

71.2 

0.00 

1900 

66.6 

0.00 

2000 

63.0 

0.00 

2100 

60.3 

0.00 

2200 

58.5 

0.00 

2300 

57.6 

0.00 

2359 

57.2 

0.00 

18  Sep 

0100 

56.8 

0.00 

0200 

56.1 

0.00 

0300 

55.0 

0.00 

0400 

54.3 

0.00 

0500 

53.8 

0.00 

0600 

54.0 

0.00 

0700 

55.4 

0.00 

0800 

64.2 

0.00 

0900 

69.4 

0.00 

1000 

72.1 

0.00 

1100 

74.5 

0.00 

1200 

76.1 

0.00 

1300 

76.8 

0.00 

1400 

77.0 

0.00 

1500 

77.5 

0.00 

1600 

76.6 

0.00 

1700 

75.4 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

18  Sep 

1800 

73.4 

0.00 

1900 

69.3 

0.00 

2000 

67.3 

0.00 

2100 

68.7 

0.00 

2200 

67.3 

0.00 

2300 

66.4 

0.00 

2359 

64.6 

0.00 

19  Sep 

0100 

64.0 

0.00 

0200 

62.2 

0.00 

0300 

60.8 

0.00 

0400 

60.1 

0.00 

0500 

59.2 

0.00 

0600 

58.1 

0.00 

0700 

58.5 

0.00 

0800 

61.3 

0.00 

0900 

63.1 

0.00 

1000 

65.7 

0.00 

1100 

67.5 

0.00 

1200 

68.5 

0.00 

1300 

69.4 

0.00 

1400 

70.0 

0.00 

1500 

70.2 

0.00 

1600 

70.3 

0.00 

1700 

68.2 

0.00 

1800 

66.9 

0.00 

1900 

63.3 

0.00 

2000 

61.9 

0.00 

2100 

61.2 

0.00 

2200 

59.9 

0.00 

2300 

58.3 

0.00 

2359 

57.4 

0.00 

20  Sep 

0100 

55.4 

0.00 

0200 

53.4 

0.00 

0300 

51.8 

0.00 

0400 

51.3 

0.00 

0500 

50.4 

0.00 

0600 

50.0 

0.00 

0700 

50.5 

0.00 

0800 

56.1 

0.00 

0900 

61.3 

0.00 

1000 

64.6 

0.00 

1100 

66.6 

0.00 

1200 

67.5 

0.00 

1300 

68.9 

0.00 

1400 

69.4 

0.00 

1500 

70.7 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

20  Sep 

1600 

71.2 

0.00 

1700 

69.4 

0.00 

1800 

67.3 

0.00 

1900 

62.4 

0.00 

2000 

59.5 

0.00 

2100 

57.7 

0.00 

2200 

55.6 

0.00 

2300 

54.1 

0.00 

2359 

53.2 

0.00 

21  Sep 

0100 

52.5 

0.00 

0200 

51.4 

0.00 

0300 

50.2 

0.00 

0400 

49.5 

0.00 

0500 

48.9 

0.00 

0600 

49.1 

0.00 

0700 

50.4 

0.00 

0800 

58.1 

0.00 

0900 

64.4 

0.00 

1000 

69.6 

0.00 

1100 

72.9 

0.00 

1200 

75.6 

0.00 

1300 

77.2 

0.00 

1400 

78.8 

0.00 

1500 

78.3 

0.00 

1600 

78.1 

0.00 

1700 

77.0 

0.00 

1800 

74.1 

0.00 

1900 

67.6 

0.00 

2000 

64.6 

0.00 

2100 

62.1 

0.00 

2200 

60.8 

0.00 

2300 

59.7 

0.00 

2359 

58.5 

0.00 

22  Sep 

0100 

57.6 

0.00 

0200 

57.0 

0.00 

0300 

56.7 

0.00 

0400 

56.1 

0.00 

0500 

59.0 

0.00 

0600 

59.2 

0.00 

0700 

59.7 

0.00 

0800 

63.5 

0.00 

0900 

67.6 

0.00 

1000 

69.8 

0.00 

1100 

72.3 

0.00 

1200 

74.5 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

22  Sep 

1300 

75.9 

0.00 

1400 

77.0 

0.00 

1500 

77.7 

0.00 

1600 

76.5 

0.00 

1700 

75.7 

0.00 

1800 

74.3 

0.00 

1900 

73.2 

0.00 

2000 

72.1 

0.00 

2100 

71.6 

0.00 

2200 

70.2 

0.00 

2300 

67.8 

0.00 

2359 

65.8 

0.00 

23  Sep 

0100 

65.3 

0.00 

0200 

64.6 

0.00 

0300 

63.7 

0.00 

0400 

61.9 

0.00 

0500 

60.4 

0.00 

0600 

59.4 

0.00 

0700 

59.5 

0.00 

0800 

61.9 

0.00 

0900 

64.0 

0.00 

1000 

66.2 

0.00 

1100 

67.8 

0.00 

1200 

69.1 

0.00 

1300 

70.3 

0.00 

1400 

71.4 

0.00 

1500 

72.1 

0.00 

1600 

72.3 

0.00 

1700 

71.4 

0.00 

1800 

68.9 

0.00 

1900 

64.2 

0.00 

2000 

59.7 

0.00 

2100 

56.8 

0.00 

2200 

57.7 

0.00 

2300 

59.0 

0.00 

2359 

55.4 

0.00 

24  Sep 

0100 

55.0 

0.00 

0200 

55.6 

0.00 

0300 

55.0 

0.00 

0400 

54.7 

0.00 

0500 

53.2 

0.00 

0600 

53.8 

0.00 

0700 

55.8 

0.00 

0800 

60.3 

0.00 

0900 

64.0 

0.00 

1000 

65.5 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

24  Sep 

1100 

68.0 

0.00 

1200 

69.4 

0.00 

1300 

70.9 

0.00 

1400 

72.0 

0.00 

1500 

71.6 

0.00 

1600 

71.4 

0.00 

1700 

70.5 

0.00 

1800 

68.5 

0.00 

1900 

67.1 

0.00 

2000 

65.7 

0.00 

2100 

64.0 

0.00 

2200 

62.4 

0.00 

2300 

61.7 

0.00 

2359 

60.8 

0.00 

25  Sep 

0100 

59.4 

0.00 

0200 

58.8 

0.00 

0300 

57.9 

0.00 

0400 

57.9 

0.00 

0500 

57.2 

0.00 

0600 

56.5 

0.00 

0700 

56.7 

0.00 

0800 

58.5 

0.00 

0900 

59.9 

0.00 

1000 

61.9 

0.00 

1100 

63.3 

0.00 

1200 

64.8 

0.00 

1300 

64.8 

0.00 

1400 

63.0 

0.02 

1500 

62.6 

0.01 

1600 

63.5 

0.00 

1700 

63.5 

0.01 

1800 

61.9 

0.01 

1900 

61.3 

0.01 

2000 

60.8 

0.03 

2100 

60.6 

0.01 

2200 

59.5 

0.03 

2300 

59.9 

0.00 

2359 

60.4 

0.00 

26  Sep 

0100 

60.6 

0.00 

0200 

61.0 

0.00 

0300 

61.7 

0.00 

0400 

62.2 

0.00 

0500 

62.4 

0.02 

0600 

62.1 

0.01 

0700 

62.4 

0.00 

0800 

63.1 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

26  Sep 

0900 

64.0 

0.00 

1000 

65.1 

0.00 

1100 

66.4 

0.00 

1200 

67.8 

0.00 

1300 

68.2 

0.00 

1400 

67.6 

0.00 

1500 

67.8 

0.00 

1600 

68.9 

0.00 

1700 

68.9 

0.00 

1800 

68.5 

0.00 

1900 

68.7 

0.00 

2000 

68.9 

0.00 

2100 

68.7 

0.00 

2200 

68.9 

0.01 

2300 

68.7 

0.01 

2359 

68.5 

0.06 

27  Sep 

0100 

68.7 

0.00 

0200 

68.9 

0.00 

0300 

69.1 

0.00 

0400 

69.3 

0.05 

0500 

69.3 

0.01 

0600 

69.4 

0.00 

0700 

70.2 

0.00 

0800 

70.5 

0.05 

0900 

71.2 

0.00 

1000 

72.0 

0.00 

1100 

72.1 

0.15 

1200 

70.3 

0.12 

1300 

71.4 

0.00 

1400 

72.0 

0.01 

1500 

72.7 

0.00 

1600 

73.8 

0.00 

1700 

73.8 

0.00 

1800 

72.3 

0.00 

1900 

72.0 

0.00 

2000 

71.2 

0.00 

2100 

70.5 

0.00 

2200 

70.0 

0.05 

2300 

69.4 

0.02 

2359 

69.4 

0.09 

28  Sep 

0100 

69.3 

0.00 

0200 

69.3 

0.00 

0300 

69.1 

0.00 

0400 

68.9 

0.00 

0500 

68.7 

0.00 

0600 

68.7 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

28  Sep 

0700 

68.9 

0.00 

0800 

69.6 

0.00 

0900 

70.2 

0.00 

1000 

72.0 

0.00 

1100 

73.4 

0.00 

1200 

76.3 

0.00 

1300 

76.8 

0.01 

1400 

73.9 

0.05 

1500 

74.8 

0.00 

1600 

72.9 

0.10 

1700 

71.2 

0.01 

1800 

70.2 

0.00 

1900 

68.7 

0.00 

2000 

68.7 

0.00 

2100 

68.5 

0.00 

2200 

68.0 

0.00 

2300 

67.5 

0.00 

2359 

67.5 

0.00 

29  Sep 

0100 

66.9 

0.00 

0200 

66.4 

0.00 

0300 

65.8 

0.00 

0400 

64.9 

0.00 

0500 

64.2 

0.00 

0600 

63.0 

0.00 

0700 

63.0 

0.00 

0800 

66.9 

0.00 

0900 

69.8 

0.00 

1000 

71.6 

0.00 

1100 

72.3 

0.00 

1200 

72.0 

0.00 

1300 

72.0 

0.00 

1400 

71.8 

0.00 

1500 

72.7 

0.00 

1600 

73.8 

0.00 

1700 

72.5 

0.00 

1800 

69.6 

0.00 

1900 

66.0 

0.00 

2000 

63.0 

0.00 

2100 

60.8 

0.00 

2200 

59.7 

0.00 

2300 

59.5 

0.00 

2359 

57.7 

0.00 

30  Sep 

0100 

55.6 

0.00 

0200 

54.3 

0.00 

0300 

53.2 

0.00 

0400 

53.1 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

30  Sep 

0500 

52.3 

0.00 

0600 

51.6 

0.00 

0700 

52.5 

0.00 

0800 

58.6 

0.00 

0900 

63.9 

0.00 

1000 

65.8 

0.00 

1100 

68.4 

0.00 

1200 

71.2 

0.00 

1300 

72.0 

0.00 

1400 

72.7 

0.00 

1500 

72.7 

0.00 

1600 

72.7 

0.00 

1700 

71.2 

0.00 

1800 

69.1 

0.00 

1900 

68.2 

0.14 

2000 

62.2 

0.13 

2100 

61.2 

0.01 

2200 

61.3 

0.05 

2300 

60.8 

0.00 

2359 

60.3 

0.00 

1  Oct 

0100 

60.1 

0.00 

0200 

59.7 

0.01 

0300 

59.2 

0.00 

0400 

58.5 

0.00 

0500 

57.2 

0.00 

0600 

55.4 

0.00 

0700 

54.9 

0.00 

0800 

57.6 

0.00 

0900 

59.9 

0.00 

1000 

63.3 

0.00 

1100 

68.0 

0.00 

1200 

70.9 

0.00 

1300 

72.1 

0.00 

1400 

66.4 

0.17 

1500 

63.3 

0.06 

1600 

62.1 

0.02 

1700 

61.2 

0.00 

1800 

60.4 

0.00 

1900 

59.7 

0.00 

2000 

59.2 

0.01 

2100 

58.3 

0.02 

2200 

57.4 

0.03 

2300 

55.9 

0.00 

2359 

53.6 

0.00 

2  Oct 

0100 

52.5 

0.00 

0200 

51.4 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

2  Oct 

0300 

50.4 

0.00 

0400 

50.0 

0.00 

0500 

49.5 

0.00 

0600 

49.6 

0.00 

0700 

50.2 

o.oo  ! 

0800 

54.1 

0.00 

0900 

57.0 

0.00  i 

1000 

59.0 

0.00 

1100 

60.8 

0.00 

1200 

61.2 

0.00 

1300 

62.2 

0.00 

1400 

62.8 

0.00 

1500 

63.0 

0.00 

1600 

62.8 

0.00 

1700 

62.2 

0.00 

1800 

59.0 

0.00 

1900 

55.6 

0.00 

2000 

50.2 

0.00 

2100 

47.8 

0.00 

2200 

47.3 

0.00 

2300 

46.6 

0.00 

2359 

45.5 

o.oo  ! 

3  Oct 

0100 

45.0 

0.00 

0200 

44.6 

0.00 

0300 

44.8 

0.00 

0400 

46.4 

0.00 

0500 

48.9 

0.00 

0600 

49.5 

0.00 

0700 

50.2 

0.00 

0800 

55.2 

0.00 

0900 

59.4 

0.00 

1000 

63.7 

0.00 

1100 

66.4 

0.00 

1200 

68.7 

0.00 

1300 

69.4 

0.00 

1400 

69.4 

0.00 

1500 

69.6 

0.00 

1600 

69.3 

0.00 

1700 

67.6 

o.oo  ! 

1800 

64.6 

0.00 

1900 

60.1 

0.00 

2000 

57.0 

0.00 

2100 

54.1 

0.00 

2200 

53.4 

0.00 

2300 

53.1 

0.00 
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Date,  08 

Time,  EST 

Avg  Temperature,  °F 

Total  Precipitation,  in. 

3  Oct 

2359 

53.6 

0.00 

4  Oct 

0100 

54.5 

0.00 

0200 

54.7 

0.00 

0300 

54.7 

0.00 

0400 

54.1 

0.00 

0500 

53.6 

0.00 

0600 

52.2 

0.00 

0700 

50.9 

0.00 

0800 

54.7 

0.00 

0900 

58.6 

0.00 

1000 

60.8 

0.00 

1100 

63.0 

0.00 

1200 

64.8 

0.00 

1300 

66.0 

0.00 

1400 

66.6 

0.00 

1500 

68.0 

0.00 

1600 

68.2 

0.00 

1700 

67.6 

0.00 

1800 

62.6 

0.00 

1900 

57.4 

0.00 

2000 

55.4 

0.00 

2100 

54.9 

0.00 

2200 

54.3 

0.00 

2300 

54.0 

0.00 

2359 

53.2 

0.00 

B-l  1 
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APPENDIX  C.  SOIL  MOISTURE 


Date:  15  Sep  08 

Times:  N/A  through  1415 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

1.7 

6  to  12 

N/A 

3.4 

12  to  24 

N/A 

5.4 

24  to  36 

N/A 

3.7 

36  to  48 

N/A 

3.7 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-l 


C-2 


Date:  17  Sep  08 

Times:  1000  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

1.7 

N/A 

6  to  12 

3.1 

N/A 

12  to  24 

5.4 

N/A 

24  to  36 

3.7 

N/A 

36  to  48 

3.7 

N/A 

Blind  grid/moguls 

0  to  6 

1.6 

1.6 

6  to  12 

3.7 

3.7 

12  to  24 

3.7 

3.7 

24  to  36 

3.7 

3.7 

36  to  48 

3.7 

3.7 

C-3 


Date:  18  Sep  08 

Times:  0700  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

1.6 

1.6 

6  to  12 

3.7 

3.7 

12  to  24 

3.7 

3.7 

24  to  36 

3.7 

3.7 

36  to  48 

3.7 

3.7 

C-4 


Date:  19  Sep  08 

Times:  0700  through  1800 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.6 

5.4 

6  to  12 

8.2 

8.1 

12  to  24 

11.9 

11.8 

24  to  36 

21.4 

21.3 

36  to  48 

21.9 

21.7 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

1.5 

N/A 

6  to  12 

3.6 

N/A 

12  to  24 

3.7 

N/A 

24  to  36 

3.7 

N/A 

36  to  48 

3.7 

N/A 

C-5 


Date:  20  Sep  08 

Times:  0700  through  1500 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-6 


Date:  22  Sep  08 

Times:  0700  through  1800 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-7 


Date:  23  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.3 

5.3 

6  to  12 

7.7 

7.6 

12  to  24 

11.3 

11.3 

24  to  36 

21.1 

21.0 

36  to  48 

21.5 

21.7 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-8 


Date:  24  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.2 

5.1 

6  to  12 

7.8 

7.7 

12  to  24 

11.2 

11.1 

24  to  36 

20.9 

20.9 

36  to  48 

21.5 

21.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-9 


Date:  25  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.0 

5.2 

6  to  12 

7.8 

7.9 

12  to  24 

11.1 

11.8 

24  to  36 

20.8 

21.6 

36  to  48 

21.5 

21.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-10 


Date:  26  Sep  08 

Times:  0700  through  1700 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.9 

5.8 

6  to  12 

8.6 

8.8 

12  to  24 

11.9 

12.4 

24  to  36 

21.9 

21.9 

36  to  48 

22.6 

22.5 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-ll 


Date:  27  Sep  08 

Times:  0700  through  1400 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

5.9 

5.8 

6  to  12 

9.6 

9.7 

12  to  24 

12.8 

12.9 

24  to  36 

22.6 

22.7 

36  to  48 

23.9 

23.8 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-12 


Date:  29  Sep  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


0  to  6 


6  to  12 


12  to  24 


24  to  36 


36  to  48 


C-13 


Date:  30  Sep  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

3.6 

3.6 

6  to  12 

3.7 

3.8 

12  to  24 

6.6 

6.5 

24  to  36 

5.3 

5.2 

36  to  48 

4.8 

4.7 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-14 


Date:  1  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.8 

6.7 

6  to  12 

10.8 

10.7 

12  to  24 

13.7 

13.6 

24  to  36 

22.5 

22.8 

36  to  48 

23.6 

23.9 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-15 


Date:  2  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.9 

6.8 

6  to  12 

10.8 

10.6 

12  to  24 

13.9 

13.8 

24  to  36 

22.7 

22.8 

36  to  48 

23.6 

23.9 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

C-16 


Date:  3  Oct  08 

Times:  0700  through  1730 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

2.3 

2.3 

6  to  12 

3.8 

3.7 

12  to  24 

3.9 

3.9 

24  to  36 

5.2 

5.1 

36  to  48 

5.7 

5.9 
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Date:  4  Oct  08 

Times:  0700  through  1330 

Probe  Location: 

Layer,  in. 

AM  Reading,  % 

PM  Reading,  % 

Wet  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Wooded  area 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Open  area 

0  to  6 

6.5 

6.5 

6  to  12 

10.8 

10.7 

12  to  24 

13.6 

13.6 

24  to  36 

22.4 

22.2 

36  to  48 

23.7 

23.6 

Calibration  lanes 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 

Blind  grid/moguls 

0  to  6 

N/A 

N/A 

6  to  12 

N/A 

N/A 

12  to  24 

N/A 

N/A 

24  to  36 

N/A 

N/A 

36  to  48 

N/A 

N/A 
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Date 

Ni 

Pe 

m 

f 

e 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Time 

Duration 

min. 

Operational  Status 

09/29/08 

3 

CALIBRATION 

LANES 

745 

1040 

175 

INITIAL  SET-UP 

09/29/08 

3 

CALIBRATION 

LANES 

1040 

1105 

25 

CALIBRATION 

09/29/08 

3 

CALIBRATION 

LANES 

1105 

1115 

10 

COLLECTING 

DATA 

09/29/08 

3 

CALIBRATION 

LANES 

1115 

1125 

10 

DOWNTIME  DUE 
TO  EQUIP 
MA1NT/CHECK 

09/29/08 

3 

CALIBRATION 

LANES 

1125 

1430 

185 

COLLECTING 

DATA 

09/29/08 

3 

CALIBRATION 

LANES 

1430 

1525 

55 

DOWNTIME  DUE 
TO  EQUIPMENT 
FAILURE 

09/29/08 

3 

CALIBRATION 

LANES 

1525 

1630 

65 

COLLECTING 

DATA 

09/29/08 

3 

CALIBRATION 

LANES 

1630 

1700 

30 

DAILY  START, 
STOP 

09/30/08 

3 

CALIBRATION 

LANES 

750 

855 

65 

DAILY  START, 
STOP 

09/30/08 

3 

CALIBRATION 

LANES 

855 

910 

15 

CALIBRATION 

09/30/08 

3 

CALIBRATION 

LANES 

910 

1045 

95 

COLLECTING 

DATA 

09/30/08 

3 

CALIBRATION 

LANES 

1045 

1105 

20 

DOWNTIME  DUE 
TO  EQUIPMENT 
FAILURE 

09/30/08 

3 

CALIBRATION 

LANES 

1105 

1410 

185 

COLLECTING 

DATA 

09/30/08 

3 

CALIBRATION 

LANES 

1410 

1420 

10 

CALIBRATION 

09/30/08 

3 

CALIBRATION 

LANES 

1420 

1430 

10 

DAILY  START, 
STOP 

Operational  Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

INITIAL  SET  UP 
AOL  ADVANCE 
ORDNANCE 
LOCATOR 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 
DATA,  1/2  METER 
LINE  SPACING 

GPS 

LINEAR 

SUNNY 

MUDDY 

CHANGE 

BATTERY 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

WOODEN  ARM 
BROKE, ATTACHE 

D  TO  TRAILER 
HITCH,  REPLACED 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

SETUP 

EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

WOODEN  ARM 
BROKE, ATTACHE 

D  TO  TRAILER 
HITCH,  REPLACED 

GPS 

LINEAR 

SUNNY 

MUDDY 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

APPENDIX  D.  DAILY  ACTIVITY  LOGS 


Date 

■1 

Area  Tested 

Status 

Start 

Time 

Status 

Stop 

Tim 

Duration 

min. 

Operational  Status 

Operational  Status  - 
Comments 

Track 

Method 

Pattern 

Field  Conditions 

10/03/08 

3 

BLIND  TEST 
GRID 

740 

1000 

140 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

BLIND  TEST 
GRID 

1000 

1010 

10 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

BLIND  TEST 
GRID 

1010 

1515 

305 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

BLIND  TEST 
GRID 

1515 

1520 

5 

CALIBRATION 

CALIBRATION 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

MINE  GRID 

1520 

1545 

25 

DAILY  START, 
STOP 

SET  UP 
EQUIPMENT 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

MINE  GRID 

1545 

1615 

30 

COLLECTING 

DATA 

COLLECTING 

DATA 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/03/08 

3 

MINE  GRID 

1615 

1635 

20 

DAILY  START, 
STOP 

EQUIPMENT 

BREAKDOWN 

GPS 

LINEAR 

SUNNY 

MUDDY 

10/04/08 

3 

MINE  GRID 

1250 

1345 

55 

DEMOBILIZATION 

DEMOBILIZATION 

GPS 

LINEAR 

SUNNY 

MUDDY 
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APPENDIX  F.  ABBREVIATIONS 


ADST 

= 

Aberdeen  Data  Services  Team 

AOL 

= 

Advanced  Ordnance  Locator 

APG 

= 

Aberdeen  Proving  Ground 

ATC 

= 

U.S.  Anny  Aberdeen  Test  Center 

BAH 

= 

Booz  Allen  Hamilton 

BAR 

= 

background  alarm  rate 

DAQ 

= 

data  acquisition 

DMM 

= 

discarded  military  munitions 

DPRT 

= 

Duke  Pattern  Recognition  Toolbox 

EM 

= 

electromagnetic 

EMI 

= 

electromagnetic  interference 

EQT 

= 

Environmental  Quality  Technology 

ERDC 

— 

U.S.  Army  Corps  of  Engineers  Engineering  Research  and 
Development  Center 

ESTCP 

= 

Environmental  Security  Technology  Certification  Program 

GPS 

= 

Global  Positioning  System 

GS 

= 

Geosoft  Script 

GT 

= 

ground  truth 

GX 

= 

Geosoft  Executable 

HDSD 

= 

Homeland  Defense  and  Sustainment  Division 

HEAT 

= 

high-explosive  antitank 

JPG 

= 

Jefferson  Proving  Ground 

MAG 

= 

magnetometer 

MM 

= 

military  numitions 

NAVEODTECHDIV 

= 

Naval  Explosive  Ordnance  Disposal  Technology  Division 

NS 

= 

nonstandard  munition 

POC 

= 

point  of  contact 

QA 

= 

quality  assurance 

QC 

= 

quality  control 

ROC 

= 

receiver-operating  characteristic 

RTK 

= 

real-time  kinematic 

S 

= 

standard  munition 

SERDP 

= 

Strategic  Environmental  Research  and  Development  Program 

TEM 

= 

time-gated  electromagnetic 

USAEC 

= 

U.S.  Army  Environmental  Command 

uxo 

= 

unexploded  ordnance 

YPG 

= 

U.S.  Army  Yuma  Proving  Ground 
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APPENDIX  G.  DISTRIBUTION  LIST 

DTC  Project  No.  8-CO-160-UXO-021 

Addressee 

No.  of 
Conies 

Commander 

U.S.  Army  Aberdeen  Test  Center 

ATTN:  TEDT-AT-SLE  (Mr.  J.  Stephen  McClung) 

400  Colleran  Road 

Aberdeen  Proving  Ground,  MD  21005-5059 

1 

Commander 

U.S.  Army  Environmental  Command 

ATTN:  IMAE-RTA  (Ms.  Kimberly  Watts) 

314  Longs  Corner  Road 

Aberdeen  Proving  Ground,  MD  21010-5401 

1 

Booz  Allen  Hamilton 

ATTN:  Mr.  Patrick  McDonnell 

4692  Millennium  Drive,  Suite  200 

Belcamp,  MD  21017-1535 

1 

G&G  Sciences,  Inc. 

ATTN:  Mr.  David  George 

23  Road 

Grand  Junction,  CO  81505 

1 

Defense  Technical  Infonnation  Center 

8725  John  J.  Kingman  Road,  Suite  0944 

Fort  Belvoir,  VA  22060-62 1 8 
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